2023-07-15 11:55:13,989 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471 2023-07-15 11:55:13,997 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-log-dir so I do NOT create it in target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b 2023-07-15 11:55:13,997 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-log-dir Erasing configuration value by system value. 2023-07-15 11:55:13,997 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-tmp-dir so I do NOT create it in target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b 2023-07-15 11:55:13,997 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-15 11:55:13,998 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b 2023-07-15 11:55:14,010 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-07-15 11:55:14,012 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL timeout: 13 mins 2023-07-15 11:55:14,040 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=215, ProcessCount=170, AvailableMemoryMB=7138 2023-07-15 11:55:14,053 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-log-dir so I do NOT create it in target/test-data/93511464-8677-cbda-8451-e447d2535500 2023-07-15 11:55:14,054 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-log-dir Erasing configuration value by system value. 2023-07-15 11:55:14,054 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-tmp-dir so I do NOT create it in target/test-data/93511464-8677-cbda-8451-e447d2535500 2023-07-15 11:55:14,054 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-15 11:55:14,055 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/93511464-8677-cbda-8451-e447d2535500 2023-07-15 11:55:14,063 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-log-dir so I do NOT create it in target/test-data/97f1c76b-5320-cd87-b893-d23729c1729c 2023-07-15 11:55:14,064 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-log-dir Erasing configuration value by system value. 2023-07-15 11:55:14,064 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-tmp-dir so I do NOT create it in target/test-data/97f1c76b-5320-cd87-b893-d23729c1729c 2023-07-15 11:55:14,064 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-15 11:55:14,064 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/97f1c76b-5320-cd87-b893-d23729c1729c 2023-07-15 11:55:14,068 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/cluster_ccea1b3f-c80a-2107-12f6-5d4a658e0622, deleteOnExit=true 2023-07-15 11:55:14,178 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/cluster_ccea1b3f-c80a-2107-12f6-5d4a658e0622/zookeeper_0, clientPort=57666, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/cluster_ccea1b3f-c80a-2107-12f6-5d4a658e0622/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/cluster_ccea1b3f-c80a-2107-12f6-5d4a658e0622/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-15 11:55:14,193 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=57666 2023-07-15 11:55:14,197 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-15 11:55:14,197 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-15 11:55:14,197 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/test.cache.data in system properties and HBase conf 2023-07-15 11:55:14,198 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop.tmp.dir in system properties and HBase conf 2023-07-15 11:55:14,198 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/hadoop.log.dir in system properties and HBase conf 2023-07-15 11:55:14,198 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-15 11:55:14,199 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-15 11:55:14,199 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-15 11:55:14,295 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-15 11:55:14,690 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-15 11:55:14,693 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-15 11:55:14,693 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-15 11:55:14,694 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-15 11:55:14,694 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-15 11:55:14,694 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-15 11:55:14,695 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-15 11:55:14,695 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-15 11:55:14,695 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-15 11:55:14,695 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-15 11:55:14,696 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/nfs.dump.dir in system properties and HBase conf 2023-07-15 11:55:14,696 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/java.io.tmpdir in system properties and HBase conf 2023-07-15 11:55:14,696 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-15 11:55:14,696 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-15 11:55:14,696 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-15 11:55:15,228 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-15 11:55:15,233 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-15 11:55:15,516 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-15 11:55:15,665 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-15 11:55:15,685 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-15 11:55:15,723 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-15 11:55:15,756 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/java.io.tmpdir/Jetty_localhost_41705_hdfs____.qob33r/webapp 2023-07-15 11:55:15,906 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41705 2023-07-15 11:55:15,923 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-15 11:55:15,924 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-15 11:55:16,448 WARN [Listener at localhost/44973] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-15 11:55:16,517 WARN [Listener at localhost/44973] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-15 11:55:16,539 WARN [Listener at localhost/44973] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-15 11:55:16,547 INFO [Listener at localhost/44973] log.Slf4jLog(67): jetty-6.1.26 2023-07-15 11:55:16,555 INFO [Listener at localhost/44973] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/java.io.tmpdir/Jetty_localhost_33477_datanode____.rc43mt/webapp 2023-07-15 11:55:16,685 INFO [Listener at localhost/44973] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33477 2023-07-15 11:55:17,027 WARN [Listener at localhost/36239] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-15 11:55:17,040 WARN [Listener at localhost/36239] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-15 11:55:17,043 WARN [Listener at localhost/36239] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-15 11:55:17,045 INFO [Listener at localhost/36239] log.Slf4jLog(67): jetty-6.1.26 2023-07-15 11:55:17,056 INFO [Listener at localhost/36239] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/java.io.tmpdir/Jetty_localhost_44229_datanode____.v5ujhc/webapp 2023-07-15 11:55:17,176 INFO [Listener at localhost/36239] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44229 2023-07-15 11:55:17,187 WARN [Listener at localhost/35553] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-15 11:55:17,554 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd197794a8ee0e20f: Processing first storage report for DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8 from datanode 1840765f-84f0-4d5b-b6a6-aff031f617fa 2023-07-15 11:55:17,556 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd197794a8ee0e20f: from storage DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8 node DatanodeRegistration(127.0.0.1:37061, datanodeUuid=1840765f-84f0-4d5b-b6a6-aff031f617fa, infoPort=42767, infoSecurePort=0, ipcPort=35553, storageInfo=lv=-57;cid=testClusterID;nsid=1805911616;c=1689422115308), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-15 11:55:17,556 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6826cf1c66abc2e1: Processing first storage report for DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e from datanode 9cf91406-fb7f-48b9-8af1-5c2d38b174cd 2023-07-15 11:55:17,557 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6826cf1c66abc2e1: from storage DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e node DatanodeRegistration(127.0.0.1:42115, datanodeUuid=9cf91406-fb7f-48b9-8af1-5c2d38b174cd, infoPort=40477, infoSecurePort=0, ipcPort=36239, storageInfo=lv=-57;cid=testClusterID;nsid=1805911616;c=1689422115308), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-15 11:55:17,557 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd197794a8ee0e20f: Processing first storage report for DS-41424662-8d64-443e-acde-a58abaf37914 from datanode 1840765f-84f0-4d5b-b6a6-aff031f617fa 2023-07-15 11:55:17,557 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd197794a8ee0e20f: from storage DS-41424662-8d64-443e-acde-a58abaf37914 node DatanodeRegistration(127.0.0.1:37061, datanodeUuid=1840765f-84f0-4d5b-b6a6-aff031f617fa, infoPort=42767, infoSecurePort=0, ipcPort=35553, storageInfo=lv=-57;cid=testClusterID;nsid=1805911616;c=1689422115308), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-15 11:55:17,558 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6826cf1c66abc2e1: Processing first storage report for DS-3ef005e8-7a51-4cfc-9577-98734e113799 from datanode 9cf91406-fb7f-48b9-8af1-5c2d38b174cd 2023-07-15 11:55:17,558 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6826cf1c66abc2e1: from storage DS-3ef005e8-7a51-4cfc-9577-98734e113799 node DatanodeRegistration(127.0.0.1:42115, datanodeUuid=9cf91406-fb7f-48b9-8af1-5c2d38b174cd, infoPort=40477, infoSecurePort=0, ipcPort=36239, storageInfo=lv=-57;cid=testClusterID;nsid=1805911616;c=1689422115308), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-15 11:55:17,633 DEBUG [Listener at localhost/35553] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471 2023-07-15 11:55:17,645 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:17,648 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:18,334 INFO [Listener at localhost/35553] util.FSUtils(471): Created version file at hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1 with version=8 2023-07-15 11:55:18,334 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/hbase-staging 2023-07-15 11:55:18,710 INFO [Listener at localhost/35553] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-15 11:55:19,259 INFO [Listener at localhost/35553] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-15 11:55:19,297 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:19,298 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:19,299 INFO [Listener at localhost/35553] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:19,299 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:19,299 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:19,460 INFO [Listener at localhost/35553] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:19,538 DEBUG [Listener at localhost/35553] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-15 11:55:19,633 INFO [Listener at localhost/35553] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38223 2023-07-15 11:55:19,645 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:19,648 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:19,676 INFO [Listener at localhost/35553] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38223 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:19,716 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:382230x0, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:19,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38223-0x10168d630520000 connected 2023-07-15 11:55:19,743 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-15 11:55:19,744 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-15 11:55:19,747 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-15 11:55:19,755 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38223 2023-07-15 11:55:19,755 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38223 2023-07-15 11:55:19,755 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38223 2023-07-15 11:55:19,756 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38223 2023-07-15 11:55:19,756 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38223 2023-07-15 11:55:19,760 INFO [Listener at localhost/35553] master.HMaster(444): hbase.rootdir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1, hbase.cluster.distributed=false 2023-07-15 11:55:19,825 INFO [Listener at localhost/35553] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-15 11:55:19,825 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:19,825 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:19,825 INFO [Listener at localhost/35553] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:19,826 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:19,826 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:19,830 INFO [Listener at localhost/35553] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:19,833 INFO [Listener at localhost/35553] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34419 2023-07-15 11:55:19,835 INFO [Listener at localhost/35553] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-15 11:55:19,841 DEBUG [Listener at localhost/35553] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-15 11:55:19,842 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:19,844 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:19,846 INFO [Listener at localhost/35553] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34419 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:19,849 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:344190x0, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:19,850 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34419-0x10168d630520001 connected 2023-07-15 11:55:19,850 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-15 11:55:19,851 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-15 11:55:19,852 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-15 11:55:19,852 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34419 2023-07-15 11:55:19,853 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34419 2023-07-15 11:55:19,854 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34419 2023-07-15 11:55:19,855 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34419 2023-07-15 11:55:19,855 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34419 2023-07-15 11:55:19,867 INFO [Listener at localhost/35553] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-15 11:55:19,867 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:19,868 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:19,868 INFO [Listener at localhost/35553] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:19,868 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:19,868 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:19,868 INFO [Listener at localhost/35553] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:19,870 INFO [Listener at localhost/35553] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35805 2023-07-15 11:55:19,870 INFO [Listener at localhost/35553] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-15 11:55:19,871 DEBUG [Listener at localhost/35553] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-15 11:55:19,872 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:19,874 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:19,876 INFO [Listener at localhost/35553] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35805 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:19,880 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:358050x0, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:19,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:35805-0x10168d630520002 connected 2023-07-15 11:55:19,881 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-15 11:55:19,882 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-15 11:55:19,882 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-15 11:55:19,883 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35805 2023-07-15 11:55:19,883 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35805 2023-07-15 11:55:19,884 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35805 2023-07-15 11:55:19,884 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35805 2023-07-15 11:55:19,884 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35805 2023-07-15 11:55:19,886 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:19,894 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-15 11:55:19,895 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:19,915 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-15 11:55:19,915 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-15 11:55:19,915 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-15 11:55:19,915 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:19,916 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-15 11:55:19,918 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,38223,1689422118510 from backup master directory 2023-07-15 11:55:19,918 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-15 11:55:19,920 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:19,921 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-15 11:55:19,921 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:19,922 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:19,924 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-15 11:55:19,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-15 11:55:20,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/hbase.id with ID: f0521d31-92f9-4caa-8fd4-4b1a00343aa2 2023-07-15 11:55:20,047 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:20,063 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:20,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3ac0596f to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:20,150 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e54f211, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:20,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:20,177 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-15 11:55:20,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:20,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:20,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store-tmp 2023-07-15 11:55:20,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:20,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-15 11:55:20,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:20,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:20,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-15 11:55:20,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:20,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:20,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:20,272 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:20,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:20,302 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38223%2C1689422118510.jenkins-hbase4.apache.org%2C38223%2C1689422118510.regiongroup-0, suffix=, logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,38223,1689422118510, archiveDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/oldWALs, maxLogs=10 2023-07-15 11:55:20,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.CommonFSUtils$DfsBuilderUtility(753): Could not find replicate method on builder; will not set replicate when creating output stream java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DistributedFileSystem$HdfsDataOutputStreamBuilder.replicate() at java.lang.Class.getMethod(Class.java:1786) at org.apache.hadoop.hbase.util.CommonFSUtils$DfsBuilderUtility.(CommonFSUtils.java:750) at org.apache.hadoop.hbase.util.CommonFSUtils.createForWal(CommonFSUtils.java:802) at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.initOutput(ProtobufLogWriter.java:102) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.FSHLogProvider.createWriter(FSHLogProvider.java:78) at org.apache.hadoop.hbase.regionserver.wal.FSHLog.createWriterInstance(FSHLog.java:307) at org.apache.hadoop.hbase.regionserver.wal.FSHLog.createWriterInstance(FSHLog.java:70) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:195) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:215) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-15 11:55:20,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,38223,1689422118510/jenkins-hbase4.apache.org%2C38223%2C1689422118510.jenkins-hbase4.apache.org%2C38223%2C1689422118510.regiongroup-0.1689422120328 2023-07-15 11:55:20,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42115,DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e,DISK], DatanodeInfoWithStorage[127.0.0.1:37061,DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8,DISK]] 2023-07-15 11:55:20,359 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:20,359 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:20,363 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:20,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:20,418 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:20,424 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-15 11:55:20,447 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-15 11:55:20,461 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:20,467 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:20,469 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:20,485 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:20,490 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-15 11:55:20,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62826091, jitterRate=-0.06381829082965851}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-15 11:55:20,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:20,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-15 11:55:20,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-15 11:55:20,518 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-15 11:55:20,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-15 11:55:20,522 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-15 11:55:20,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-07-15 11:55:20,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-15 11:55:20,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-15 11:55:20,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-15 11:55:20,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-15 11:55:20,615 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-15 11:55:20,616 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-15 11:55:20,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-15 11:55:20,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-15 11:55:20,629 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:20,631 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-15 11:55:20,631 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-15 11:55:20,643 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-15 11:55:20,647 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-15 11:55:20,647 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-15 11:55:20,647 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-15 11:55:20,647 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:20,648 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38223,1689422118510, sessionid=0x10168d630520000, setting cluster-up flag (Was=false) 2023-07-15 11:55:20,662 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:20,669 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-15 11:55:20,670 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:20,676 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:20,681 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-15 11:55:20,682 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:20,685 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.hbase-snapshot/.tmp 2023-07-15 11:55:20,687 INFO [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(951): ClusterId : f0521d31-92f9-4caa-8fd4-4b1a00343aa2 2023-07-15 11:55:20,687 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(951): ClusterId : f0521d31-92f9-4caa-8fd4-4b1a00343aa2 2023-07-15 11:55:20,692 DEBUG [RS:1;jenkins-hbase4:35805] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-15 11:55:20,692 DEBUG [RS:0;jenkins-hbase4:34419] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-15 11:55:20,699 DEBUG [RS:1;jenkins-hbase4:35805] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-15 11:55:20,699 DEBUG [RS:1;jenkins-hbase4:35805] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-15 11:55:20,705 DEBUG [RS:0;jenkins-hbase4:34419] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-15 11:55:20,705 DEBUG [RS:0;jenkins-hbase4:34419] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-15 11:55:20,707 DEBUG [RS:1;jenkins-hbase4:35805] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-15 11:55:20,708 DEBUG [RS:0;jenkins-hbase4:34419] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-15 11:55:20,709 DEBUG [RS:1;jenkins-hbase4:35805] zookeeper.ReadOnlyZKClient(139): Connect 0x5b780f41 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:20,709 DEBUG [RS:0;jenkins-hbase4:34419] zookeeper.ReadOnlyZKClient(139): Connect 0x599aae77 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:20,727 DEBUG [RS:1;jenkins-hbase4:35805] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@55bfea5e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:20,727 DEBUG [RS:1;jenkins-hbase4:35805] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@51806431, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:20,728 DEBUG [RS:0;jenkins-hbase4:34419] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@575d4b34, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:20,728 DEBUG [RS:0;jenkins-hbase4:34419] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5362c507, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:20,754 DEBUG [RS:1;jenkins-hbase4:35805] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:35805 2023-07-15 11:55:20,757 INFO [RS:1;jenkins-hbase4:35805] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-15 11:55:20,757 INFO [RS:1;jenkins-hbase4:35805] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-15 11:55:20,758 DEBUG [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(1022): About to register with Master. 2023-07-15 11:55:20,758 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:34419 2023-07-15 11:55:20,759 INFO [RS:0;jenkins-hbase4:34419] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-15 11:55:20,759 INFO [RS:0;jenkins-hbase4:34419] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-15 11:55:20,760 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1022): About to register with Master. 2023-07-15 11:55:20,761 INFO [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38223,1689422118510 with isa=jenkins-hbase4.apache.org/172.31.14.131:35805, startcode=1689422119867 2023-07-15 11:55:20,761 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38223,1689422118510 with isa=jenkins-hbase4.apache.org/172.31.14.131:34419, startcode=1689422119824 2023-07-15 11:55:20,780 DEBUG [RS:1;jenkins-hbase4:35805] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-15 11:55:20,783 DEBUG [RS:0;jenkins-hbase4:34419] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-15 11:55:20,820 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-15 11:55:20,843 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:20,843 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:20,845 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:20,845 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:20,845 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-15 11:55:20,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:20,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:20,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:20,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689422150859 2023-07-15 11:55:20,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-15 11:55:20,867 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-15 11:55:20,868 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-15 11:55:20,874 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:20,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-15 11:55:20,889 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-15 11:55:20,889 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-15 11:55:20,890 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-15 11:55:20,890 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-15 11:55:20,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:20,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-15 11:55:20,906 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-15 11:55:20,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-15 11:55:20,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-15 11:55:20,910 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-15 11:55:20,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422120913,5,FailOnTimeoutGroup] 2023-07-15 11:55:20,922 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422120922,5,FailOnTimeoutGroup] 2023-07-15 11:55:20,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:20,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-15 11:55:20,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:20,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:20,943 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-15 11:55:20,944 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-15 11:55:20,945 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1 2023-07-15 11:55:20,978 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:20,981 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-15 11:55:20,982 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43021, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-15 11:55:20,982 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46525, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-15 11:55:20,985 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/info 2023-07-15 11:55:20,986 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-15 11:55:20,987 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:20,987 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-15 11:55:20,990 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:20,990 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-15 11:55:20,991 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:20,992 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-15 11:55:20,994 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/table 2023-07-15 11:55:20,994 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-15 11:55:20,995 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38223] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:20,996 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:20,998 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740 2023-07-15 11:55:20,999 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740 2023-07-15 11:55:21,003 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-15 11:55:21,005 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-15 11:55:21,009 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-15 11:55:21,010 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73933259, jitterRate=0.10169140994548798}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-15 11:55:21,010 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-15 11:55:21,011 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-15 11:55:21,011 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-15 11:55:21,011 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38223] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:21,011 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-15 11:55:21,011 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-15 11:55:21,011 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-15 11:55:21,013 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-15 11:55:21,014 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-15 11:55:21,019 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1 2023-07-15 11:55:21,019 DEBUG [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1 2023-07-15 11:55:21,020 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:44973 2023-07-15 11:55:21,020 DEBUG [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:44973 2023-07-15 11:55:21,020 DEBUG [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-15 11:55:21,020 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-15 11:55:21,020 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-15 11:55:21,020 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-15 11:55:21,027 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-15 11:55:21,028 DEBUG [RS:1;jenkins-hbase4:35805] zookeeper.ZKUtil(162): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:21,028 DEBUG [RS:0;jenkins-hbase4:34419] zookeeper.ZKUtil(162): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:21,028 WARN [RS:0;jenkins-hbase4:34419] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:21,028 WARN [RS:1;jenkins-hbase4:35805] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:21,030 INFO [RS:0;jenkins-hbase4:34419] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:21,031 INFO [RS:0;jenkins-hbase4:34419] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:21,031 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1948): logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:21,033 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35805,1689422119867] 2023-07-15 11:55:21,033 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34419,1689422119824] 2023-07-15 11:55:21,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-15 11:55:21,031 INFO [RS:1;jenkins-hbase4:35805] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:21,037 INFO [RS:1;jenkins-hbase4:35805] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:21,039 DEBUG [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(1948): logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:21,050 DEBUG [RS:1;jenkins-hbase4:35805] zookeeper.ZKUtil(162): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:21,050 DEBUG [RS:0;jenkins-hbase4:34419] zookeeper.ZKUtil(162): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:21,050 DEBUG [RS:1;jenkins-hbase4:35805] zookeeper.ZKUtil(162): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:21,051 DEBUG [RS:0;jenkins-hbase4:34419] zookeeper.ZKUtil(162): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:21,053 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-15 11:55:21,056 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-15 11:55:21,064 DEBUG [RS:1;jenkins-hbase4:35805] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:55:21,064 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:55:21,073 INFO [RS:1;jenkins-hbase4:35805] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-15 11:55:21,073 INFO [RS:0;jenkins-hbase4:34419] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-15 11:55:21,098 INFO [RS:1;jenkins-hbase4:35805] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-15 11:55:21,098 INFO [RS:0;jenkins-hbase4:34419] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-15 11:55:21,103 INFO [RS:1;jenkins-hbase4:35805] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-15 11:55:21,103 INFO [RS:0;jenkins-hbase4:34419] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-15 11:55:21,103 INFO [RS:1;jenkins-hbase4:35805] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,103 INFO [RS:0;jenkins-hbase4:34419] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,104 INFO [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-15 11:55:21,104 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-15 11:55:21,113 INFO [RS:1;jenkins-hbase4:35805] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,113 INFO [RS:0;jenkins-hbase4:34419] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,114 DEBUG [RS:1;jenkins-hbase4:35805] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,114 DEBUG [RS:0;jenkins-hbase4:34419] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,114 DEBUG [RS:1;jenkins-hbase4:35805] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,114 DEBUG [RS:0;jenkins-hbase4:34419] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,115 DEBUG [RS:1;jenkins-hbase4:35805] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,115 DEBUG [RS:0;jenkins-hbase4:34419] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,115 DEBUG [RS:1;jenkins-hbase4:35805] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,115 DEBUG [RS:0;jenkins-hbase4:34419] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,115 DEBUG [RS:1;jenkins-hbase4:35805] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,115 DEBUG [RS:0;jenkins-hbase4:34419] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,115 DEBUG [RS:1;jenkins-hbase4:35805] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:21,115 DEBUG [RS:0;jenkins-hbase4:34419] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:21,115 DEBUG [RS:1;jenkins-hbase4:35805] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,115 DEBUG [RS:0;jenkins-hbase4:34419] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,116 DEBUG [RS:1;jenkins-hbase4:35805] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,116 DEBUG [RS:0;jenkins-hbase4:34419] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,116 DEBUG [RS:1;jenkins-hbase4:35805] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,116 DEBUG [RS:0;jenkins-hbase4:34419] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,116 DEBUG [RS:1;jenkins-hbase4:35805] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,116 DEBUG [RS:0;jenkins-hbase4:34419] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:21,118 INFO [RS:1;jenkins-hbase4:35805] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,119 INFO [RS:1;jenkins-hbase4:35805] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,119 INFO [RS:1;jenkins-hbase4:35805] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,122 INFO [RS:0;jenkins-hbase4:34419] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,122 INFO [RS:0;jenkins-hbase4:34419] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,123 INFO [RS:0;jenkins-hbase4:34419] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,135 INFO [RS:1;jenkins-hbase4:35805] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-15 11:55:21,136 INFO [RS:0;jenkins-hbase4:34419] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-15 11:55:21,138 INFO [RS:0;jenkins-hbase4:34419] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34419,1689422119824-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,138 INFO [RS:1;jenkins-hbase4:35805] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35805,1689422119867-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,157 INFO [RS:1;jenkins-hbase4:35805] regionserver.Replication(203): jenkins-hbase4.apache.org,35805,1689422119867 started 2023-07-15 11:55:21,157 INFO [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,35805,1689422119867, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35805, sessionid=0x10168d630520002 2023-07-15 11:55:21,157 DEBUG [RS:1;jenkins-hbase4:35805] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-15 11:55:21,157 DEBUG [RS:1;jenkins-hbase4:35805] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:21,157 DEBUG [RS:1;jenkins-hbase4:35805] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35805,1689422119867' 2023-07-15 11:55:21,157 DEBUG [RS:1;jenkins-hbase4:35805] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-15 11:55:21,158 DEBUG [RS:1;jenkins-hbase4:35805] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-15 11:55:21,159 INFO [RS:0;jenkins-hbase4:34419] regionserver.Replication(203): jenkins-hbase4.apache.org,34419,1689422119824 started 2023-07-15 11:55:21,159 DEBUG [RS:1;jenkins-hbase4:35805] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-15 11:55:21,159 DEBUG [RS:1;jenkins-hbase4:35805] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-15 11:55:21,159 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34419,1689422119824, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34419, sessionid=0x10168d630520001 2023-07-15 11:55:21,159 DEBUG [RS:1;jenkins-hbase4:35805] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:21,159 DEBUG [RS:1;jenkins-hbase4:35805] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35805,1689422119867' 2023-07-15 11:55:21,159 DEBUG [RS:1;jenkins-hbase4:35805] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-15 11:55:21,159 DEBUG [RS:0;jenkins-hbase4:34419] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-15 11:55:21,159 DEBUG [RS:0;jenkins-hbase4:34419] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:21,160 DEBUG [RS:0;jenkins-hbase4:34419] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34419,1689422119824' 2023-07-15 11:55:21,161 DEBUG [RS:0;jenkins-hbase4:34419] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-15 11:55:21,161 DEBUG [RS:1;jenkins-hbase4:35805] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-15 11:55:21,161 DEBUG [RS:0;jenkins-hbase4:34419] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-15 11:55:21,161 DEBUG [RS:1;jenkins-hbase4:35805] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-15 11:55:21,161 INFO [RS:1;jenkins-hbase4:35805] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-15 11:55:21,162 INFO [RS:1;jenkins-hbase4:35805] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-15 11:55:21,162 DEBUG [RS:0;jenkins-hbase4:34419] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-15 11:55:21,162 DEBUG [RS:0;jenkins-hbase4:34419] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-15 11:55:21,162 DEBUG [RS:0;jenkins-hbase4:34419] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:21,162 DEBUG [RS:0;jenkins-hbase4:34419] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34419,1689422119824' 2023-07-15 11:55:21,162 DEBUG [RS:0;jenkins-hbase4:34419] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-15 11:55:21,163 DEBUG [RS:0;jenkins-hbase4:34419] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-15 11:55:21,164 DEBUG [RS:0;jenkins-hbase4:34419] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-15 11:55:21,164 INFO [RS:0;jenkins-hbase4:34419] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-15 11:55:21,164 INFO [RS:0;jenkins-hbase4:34419] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-15 11:55:21,208 DEBUG [jenkins-hbase4:38223] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-07-15 11:55:21,212 DEBUG [jenkins-hbase4:38223] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-15 11:55:21,218 DEBUG [jenkins-hbase4:38223] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-15 11:55:21,218 DEBUG [jenkins-hbase4:38223] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-15 11:55:21,218 DEBUG [jenkins-hbase4:38223] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-15 11:55:21,221 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34419,1689422119824, state=OPENING 2023-07-15 11:55:21,231 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-15 11:55:21,233 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:21,233 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-15 11:55:21,239 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34419,1689422119824}] 2023-07-15 11:55:21,269 INFO [RS:1;jenkins-hbase4:35805] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:21,269 INFO [RS:0;jenkins-hbase4:34419] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:21,273 INFO [RS:0;jenkins-hbase4:34419] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0, suffix=, logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824, archiveDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs, maxLogs=10 2023-07-15 11:55:21,273 INFO [RS:1;jenkins-hbase4:35805] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0, suffix=, logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867, archiveDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs, maxLogs=10 2023-07-15 11:55:21,326 INFO [RS:1;jenkins-hbase4:35805] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867/jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0.1689422121276 2023-07-15 11:55:21,326 INFO [RS:0;jenkins-hbase4:34419] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276 2023-07-15 11:55:21,327 DEBUG [RS:1;jenkins-hbase4:35805] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42115,DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e,DISK], DatanodeInfoWithStorage[127.0.0.1:37061,DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8,DISK]] 2023-07-15 11:55:21,327 DEBUG [RS:0;jenkins-hbase4:34419] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37061,DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8,DISK], DatanodeInfoWithStorage[127.0.0.1:42115,DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e,DISK]] 2023-07-15 11:55:21,433 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:21,435 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:21,440 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56370, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-15 11:55:21,457 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-15 11:55:21,457 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:21,458 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:21,458 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:21,461 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34419%2C1689422119824.meta, suffix=.meta, logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824, archiveDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs, maxLogs=10 2023-07-15 11:55:21,479 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.meta.1689422121463.meta 2023-07-15 11:55:21,479 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42115,DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e,DISK], DatanodeInfoWithStorage[127.0.0.1:37061,DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8,DISK]] 2023-07-15 11:55:21,479 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:21,481 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-15 11:55:21,497 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-15 11:55:21,502 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-15 11:55:21,507 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-15 11:55:21,507 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:21,508 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-15 11:55:21,508 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-15 11:55:21,511 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-15 11:55:21,513 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/info 2023-07-15 11:55:21,513 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/info 2023-07-15 11:55:21,514 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-15 11:55:21,515 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:21,515 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-15 11:55:21,517 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:21,517 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:21,518 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-15 11:55:21,519 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:21,519 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-15 11:55:21,520 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/table 2023-07-15 11:55:21,521 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/table 2023-07-15 11:55:21,521 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-15 11:55:21,522 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:21,524 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740 2023-07-15 11:55:21,528 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740 2023-07-15 11:55:21,531 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-15 11:55:21,533 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-15 11:55:21,535 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69843617, jitterRate=0.04075099527835846}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-15 11:55:21,535 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-15 11:55:21,550 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689422121423 2023-07-15 11:55:21,570 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-15 11:55:21,571 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-15 11:55:21,571 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34419,1689422119824, state=OPEN 2023-07-15 11:55:21,575 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-15 11:55:21,575 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-15 11:55:21,581 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-15 11:55:21,582 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34419,1689422119824 in 336 msec 2023-07-15 11:55:21,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-15 11:55:21,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 549 msec 2023-07-15 11:55:21,594 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 850 msec 2023-07-15 11:55:21,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689422121595, completionTime=-1 2023-07-15 11:55:21,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-07-15 11:55:21,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-15 11:55:21,658 DEBUG [hconnection-0x16de8418-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:21,662 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56386, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:21,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-07-15 11:55:21,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689422181682 2023-07-15 11:55:21,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689422241682 2023-07-15 11:55:21,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 87 msec 2023-07-15 11:55:21,706 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38223,1689422118510-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,707 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38223,1689422118510-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,707 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38223,1689422118510-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,709 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38223, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,709 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:21,716 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-15 11:55:21,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-15 11:55:21,726 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:21,737 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-15 11:55:21,740 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-15 11:55:21,742 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-15 11:55:21,768 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:21,770 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378 empty. 2023-07-15 11:55:21,771 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:21,771 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-15 11:55:21,805 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-15 11:55:21,807 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => f1407b2bc354da8a7c08c95abb9ab378, NAME => 'hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp 2023-07-15 11:55:21,823 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:21,824 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing f1407b2bc354da8a7c08c95abb9ab378, disabling compactions & flushes 2023-07-15 11:55:21,824 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:21,824 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:21,824 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. after waiting 0 ms 2023-07-15 11:55:21,824 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:21,824 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:21,824 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for f1407b2bc354da8a7c08c95abb9ab378: 2023-07-15 11:55:21,828 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-15 11:55:21,843 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422121831"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689422121831"}]},"ts":"1689422121831"} 2023-07-15 11:55:21,869 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-15 11:55:21,873 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-15 11:55:21,877 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422121873"}]},"ts":"1689422121873"} 2023-07-15 11:55:21,881 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-15 11:55:21,886 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-15 11:55:21,889 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-15 11:55:21,889 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-15 11:55:21,889 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-15 11:55:21,892 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f1407b2bc354da8a7c08c95abb9ab378, ASSIGN}] 2023-07-15 11:55:21,894 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f1407b2bc354da8a7c08c95abb9ab378, ASSIGN 2023-07-15 11:55:21,896 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=f1407b2bc354da8a7c08c95abb9ab378, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34419,1689422119824; forceNewPlan=false, retain=false 2023-07-15 11:55:22,049 INFO [jenkins-hbase4:38223] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-15 11:55:22,050 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f1407b2bc354da8a7c08c95abb9ab378, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:22,050 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422122050"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422122050"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422122050"}]},"ts":"1689422122050"} 2023-07-15 11:55:22,056 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure f1407b2bc354da8a7c08c95abb9ab378, server=jenkins-hbase4.apache.org,34419,1689422119824}] 2023-07-15 11:55:22,221 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:22,221 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:22,226 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1, suffix=, logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824, archiveDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs, maxLogs=10 2023-07-15 11:55:22,247 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228 2023-07-15 11:55:22,248 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42115,DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e,DISK], DatanodeInfoWithStorage[127.0.0.1:37061,DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8,DISK]] 2023-07-15 11:55:22,248 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f1407b2bc354da8a7c08c95abb9ab378, NAME => 'hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:22,249 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:22,249 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:22,249 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:22,249 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:22,255 INFO [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:22,258 DEBUG [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/info 2023-07-15 11:55:22,258 DEBUG [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/info 2023-07-15 11:55:22,259 INFO [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1407b2bc354da8a7c08c95abb9ab378 columnFamilyName info 2023-07-15 11:55:22,259 INFO [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] regionserver.HStore(310): Store=f1407b2bc354da8a7c08c95abb9ab378/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:22,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:22,262 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:22,267 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:22,272 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-15 11:55:22,273 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f1407b2bc354da8a7c08c95abb9ab378; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70916918, jitterRate=0.05674442648887634}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-15 11:55:22,273 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f1407b2bc354da8a7c08c95abb9ab378: 2023-07-15 11:55:22,276 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378., pid=6, masterSystemTime=1689422122209 2023-07-15 11:55:22,280 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:22,280 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:22,288 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f1407b2bc354da8a7c08c95abb9ab378, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:22,289 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422122287"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422122287"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422122287"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422122287"}]},"ts":"1689422122287"} 2023-07-15 11:55:22,300 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-15 11:55:22,300 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure f1407b2bc354da8a7c08c95abb9ab378, server=jenkins-hbase4.apache.org,34419,1689422119824 in 239 msec 2023-07-15 11:55:22,305 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-15 11:55:22,306 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=f1407b2bc354da8a7c08c95abb9ab378, ASSIGN in 408 msec 2023-07-15 11:55:22,307 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-15 11:55:22,308 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422122307"}]},"ts":"1689422122307"} 2023-07-15 11:55:22,311 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-15 11:55:22,315 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-15 11:55:22,318 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 588 msec 2023-07-15 11:55:22,340 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-07-15 11:55:22,341 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-07-15 11:55:22,342 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:22,381 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-15 11:55:22,411 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-15 11:55:22,418 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 45 msec 2023-07-15 11:55:22,428 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-15 11:55:22,440 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-15 11:55:22,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-07-15 11:55:22,456 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-15 11:55:22,458 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-15 11:55:22,458 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.536sec 2023-07-15 11:55:22,461 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-15 11:55:22,463 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-15 11:55:22,463 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-15 11:55:22,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38223,1689422118510-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-15 11:55:22,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38223,1689422118510-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-15 11:55:22,475 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-15 11:55:22,495 DEBUG [Listener at localhost/35553] zookeeper.ReadOnlyZKClient(139): Connect 0x328bb04a to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:22,501 DEBUG [Listener at localhost/35553] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@153a1e7a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:22,518 DEBUG [hconnection-0x2d0f3cdb-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:22,532 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56402, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:22,545 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:22,545 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-15 11:55:22,546 INFO [Listener at localhost/35553] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/cluster_a703bc18-c99b-2972-822e-fef9de60bd53, deleteOnExit=true 2023-07-15 11:55:22,546 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-15 11:55:22,558 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/test.cache.data in system properties and HBase conf 2023-07-15 11:55:22,559 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/hadoop.tmp.dir in system properties and HBase conf 2023-07-15 11:55:22,559 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/hadoop.log.dir in system properties and HBase conf 2023-07-15 11:55:22,559 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-15 11:55:22,559 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-15 11:55:22,559 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-15 11:55:22,559 DEBUG [Listener at localhost/35553] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-15 11:55:22,560 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-15 11:55:22,560 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-15 11:55:22,560 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-15 11:55:22,560 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-15 11:55:22,560 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-15 11:55:22,560 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-15 11:55:22,560 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-15 11:55:22,560 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-15 11:55:22,561 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-15 11:55:22,561 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/nfs.dump.dir in system properties and HBase conf 2023-07-15 11:55:22,561 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/java.io.tmpdir in system properties and HBase conf 2023-07-15 11:55:22,561 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-15 11:55:22,561 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-15 11:55:22,561 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-15 11:55:22,573 WARN [Listener at localhost/35553] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-15 11:55:22,574 WARN [Listener at localhost/35553] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-15 11:55:22,633 WARN [Listener at localhost/35553] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-15 11:55:22,636 INFO [Listener at localhost/35553] log.Slf4jLog(67): jetty-6.1.26 2023-07-15 11:55:22,644 INFO [Listener at localhost/35553] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/java.io.tmpdir/Jetty_localhost_44583_hdfs____1x31ui/webapp 2023-07-15 11:55:22,767 INFO [Listener at localhost/35553] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44583 2023-07-15 11:55:22,780 WARN [Listener at localhost/35553] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-15 11:55:22,781 WARN [Listener at localhost/35553] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-15 11:55:22,846 WARN [Listener at localhost/46809] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-15 11:55:22,868 WARN [Listener at localhost/46809] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-15 11:55:22,872 WARN [Listener at localhost/46809] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-15 11:55:22,874 INFO [Listener at localhost/46809] log.Slf4jLog(67): jetty-6.1.26 2023-07-15 11:55:22,880 INFO [Listener at localhost/46809] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/java.io.tmpdir/Jetty_localhost_40471_datanode____.ljiltz/webapp 2023-07-15 11:55:22,985 INFO [Listener at localhost/46809] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40471 2023-07-15 11:55:22,997 WARN [Listener at localhost/33517] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-15 11:55:23,069 WARN [Listener at localhost/33517] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-15 11:55:23,072 WARN [Listener at localhost/33517] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-15 11:55:23,074 INFO [Listener at localhost/33517] log.Slf4jLog(67): jetty-6.1.26 2023-07-15 11:55:23,080 INFO [Listener at localhost/33517] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/java.io.tmpdir/Jetty_localhost_44553_datanode____.nd65ja/webapp 2023-07-15 11:55:23,179 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x62edca3e89f63454: Processing first storage report for DS-274df9f8-c308-48fc-8592-c230a3934cdd from datanode 10650aa3-cc51-41e1-84ee-1e33ce8c7eaf 2023-07-15 11:55:23,180 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x62edca3e89f63454: from storage DS-274df9f8-c308-48fc-8592-c230a3934cdd node DatanodeRegistration(127.0.0.1:33417, datanodeUuid=10650aa3-cc51-41e1-84ee-1e33ce8c7eaf, infoPort=35801, infoSecurePort=0, ipcPort=33517, storageInfo=lv=-57;cid=testClusterID;nsid=1152084081;c=1689422122581), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-15 11:55:23,180 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x62edca3e89f63454: Processing first storage report for DS-015f5a3c-e9c7-437e-9791-8e59c19b7153 from datanode 10650aa3-cc51-41e1-84ee-1e33ce8c7eaf 2023-07-15 11:55:23,180 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x62edca3e89f63454: from storage DS-015f5a3c-e9c7-437e-9791-8e59c19b7153 node DatanodeRegistration(127.0.0.1:33417, datanodeUuid=10650aa3-cc51-41e1-84ee-1e33ce8c7eaf, infoPort=35801, infoSecurePort=0, ipcPort=33517, storageInfo=lv=-57;cid=testClusterID;nsid=1152084081;c=1689422122581), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-15 11:55:23,197 INFO [Listener at localhost/33517] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44553 2023-07-15 11:55:23,207 WARN [Listener at localhost/43801] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-15 11:55:23,229 WARN [Listener at localhost/43801] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-15 11:55:23,232 WARN [Listener at localhost/43801] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-15 11:55:23,234 INFO [Listener at localhost/43801] log.Slf4jLog(67): jetty-6.1.26 2023-07-15 11:55:23,238 INFO [Listener at localhost/43801] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/java.io.tmpdir/Jetty_localhost_41363_datanode____erpg56/webapp 2023-07-15 11:55:23,354 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x87819f372b69258c: Processing first storage report for DS-50ad537a-587c-429a-875c-bc84b44b93df from datanode 0c0b17eb-6c29-4bb5-bc81-bfa1e5e20d5c 2023-07-15 11:55:23,355 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x87819f372b69258c: from storage DS-50ad537a-587c-429a-875c-bc84b44b93df node DatanodeRegistration(127.0.0.1:39035, datanodeUuid=0c0b17eb-6c29-4bb5-bc81-bfa1e5e20d5c, infoPort=36967, infoSecurePort=0, ipcPort=43801, storageInfo=lv=-57;cid=testClusterID;nsid=1152084081;c=1689422122581), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-15 11:55:23,355 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x87819f372b69258c: Processing first storage report for DS-b020c579-454e-4d5c-858f-0c0f0f769ed7 from datanode 0c0b17eb-6c29-4bb5-bc81-bfa1e5e20d5c 2023-07-15 11:55:23,355 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x87819f372b69258c: from storage DS-b020c579-454e-4d5c-858f-0c0f0f769ed7 node DatanodeRegistration(127.0.0.1:39035, datanodeUuid=0c0b17eb-6c29-4bb5-bc81-bfa1e5e20d5c, infoPort=36967, infoSecurePort=0, ipcPort=43801, storageInfo=lv=-57;cid=testClusterID;nsid=1152084081;c=1689422122581), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-15 11:55:23,374 INFO [Listener at localhost/43801] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41363 2023-07-15 11:55:23,387 WARN [Listener at localhost/32999] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-15 11:55:23,414 WARN [Listener at localhost/32999] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-15 11:55:23,416 WARN [Listener at localhost/32999] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-15 11:55:23,418 INFO [Listener at localhost/32999] log.Slf4jLog(67): jetty-6.1.26 2023-07-15 11:55:23,425 INFO [Listener at localhost/32999] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/java.io.tmpdir/Jetty_localhost_37183_datanode____h7cflb/webapp 2023-07-15 11:55:23,515 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x554ae0cc54faf87: Processing first storage report for DS-caa7d3ae-4d76-4779-b495-05a972fbb935 from datanode cdca35cf-6686-4a6c-a077-f1918b45e3da 2023-07-15 11:55:23,515 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x554ae0cc54faf87: from storage DS-caa7d3ae-4d76-4779-b495-05a972fbb935 node DatanodeRegistration(127.0.0.1:45131, datanodeUuid=cdca35cf-6686-4a6c-a077-f1918b45e3da, infoPort=39521, infoSecurePort=0, ipcPort=32999, storageInfo=lv=-57;cid=testClusterID;nsid=1152084081;c=1689422122581), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-15 11:55:23,515 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x554ae0cc54faf87: Processing first storage report for DS-7dc57bf1-f854-4979-a8c0-0719b7200ee6 from datanode cdca35cf-6686-4a6c-a077-f1918b45e3da 2023-07-15 11:55:23,515 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x554ae0cc54faf87: from storage DS-7dc57bf1-f854-4979-a8c0-0719b7200ee6 node DatanodeRegistration(127.0.0.1:45131, datanodeUuid=cdca35cf-6686-4a6c-a077-f1918b45e3da, infoPort=39521, infoSecurePort=0, ipcPort=32999, storageInfo=lv=-57;cid=testClusterID;nsid=1152084081;c=1689422122581), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-15 11:55:23,569 INFO [Listener at localhost/32999] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37183 2023-07-15 11:55:23,579 WARN [Listener at localhost/46025] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-15 11:55:23,698 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfe1a63216149c5c8: Processing first storage report for DS-f1a7c77d-1665-438f-b881-6671c08d57e3 from datanode a69a4743-a0c3-4c29-9dda-d3423c7addd2 2023-07-15 11:55:23,698 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfe1a63216149c5c8: from storage DS-f1a7c77d-1665-438f-b881-6671c08d57e3 node DatanodeRegistration(127.0.0.1:45491, datanodeUuid=a69a4743-a0c3-4c29-9dda-d3423c7addd2, infoPort=36255, infoSecurePort=0, ipcPort=46025, storageInfo=lv=-57;cid=testClusterID;nsid=1152084081;c=1689422122581), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-15 11:55:23,699 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfe1a63216149c5c8: Processing first storage report for DS-e8f35801-7d75-4cc0-a3bb-745b78c74c00 from datanode a69a4743-a0c3-4c29-9dda-d3423c7addd2 2023-07-15 11:55:23,699 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfe1a63216149c5c8: from storage DS-e8f35801-7d75-4cc0-a3bb-745b78c74c00 node DatanodeRegistration(127.0.0.1:45491, datanodeUuid=a69a4743-a0c3-4c29-9dda-d3423c7addd2, infoPort=36255, infoSecurePort=0, ipcPort=46025, storageInfo=lv=-57;cid=testClusterID;nsid=1152084081;c=1689422122581), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-15 11:55:23,793 DEBUG [Listener at localhost/46025] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b 2023-07-15 11:55:23,793 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,794 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,832 INFO [Listener at localhost/46025] util.FSUtils(471): Created version file at hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346 with version=8 2023-07-15 11:55:23,832 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/hbase-staging 2023-07-15 11:55:23,835 INFO [Listener at localhost/46025] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-15 11:55:23,835 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,835 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,835 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:23,836 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,836 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:23,836 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:23,837 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36081 2023-07-15 11:55:23,838 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,839 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,840 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=master:36081 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:23,843 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:360810x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:23,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:36081-0x10168d630520007 connected 2023-07-15 11:55:23,857 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:23,857 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:23,858 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-15 11:55:23,858 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36081 2023-07-15 11:55:23,858 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36081 2023-07-15 11:55:23,860 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36081 2023-07-15 11:55:23,862 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36081 2023-07-15 11:55:23,863 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36081 2023-07-15 11:55:23,863 INFO [Listener at localhost/46025] master.HMaster(444): hbase.rootdir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346, hbase.cluster.distributed=false 2023-07-15 11:55:23,877 INFO [Listener at localhost/46025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-15 11:55:23,878 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,878 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,878 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:23,878 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,878 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:23,878 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:23,880 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39401 2023-07-15 11:55:23,880 INFO [Listener at localhost/46025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-15 11:55:23,882 DEBUG [Listener at localhost/46025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-15 11:55:23,882 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,883 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,884 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39401 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:23,889 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:394010x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:23,890 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:394010x0, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:23,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:39401-0x10168d630520008 connected 2023-07-15 11:55:23,891 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:23,891 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-15 11:55:23,892 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39401 2023-07-15 11:55:23,892 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39401 2023-07-15 11:55:23,892 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39401 2023-07-15 11:55:23,893 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39401 2023-07-15 11:55:23,893 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39401 2023-07-15 11:55:23,905 INFO [Listener at localhost/46025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-15 11:55:23,905 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,905 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,905 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:23,905 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,905 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:23,905 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:23,907 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46827 2023-07-15 11:55:23,907 INFO [Listener at localhost/46025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-15 11:55:23,908 DEBUG [Listener at localhost/46025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-15 11:55:23,909 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,910 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,911 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46827 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:23,914 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:468270x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:23,916 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:468270x0, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:23,916 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46827-0x10168d630520009 connected 2023-07-15 11:55:23,917 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:23,917 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-15 11:55:23,917 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46827 2023-07-15 11:55:23,918 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46827 2023-07-15 11:55:23,918 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46827 2023-07-15 11:55:23,918 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46827 2023-07-15 11:55:23,919 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46827 2023-07-15 11:55:23,930 INFO [Listener at localhost/46025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-15 11:55:23,930 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,930 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,930 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:23,930 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,930 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:23,930 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:23,932 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45535 2023-07-15 11:55:23,932 INFO [Listener at localhost/46025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-15 11:55:23,933 DEBUG [Listener at localhost/46025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-15 11:55:23,934 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,935 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,936 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45535 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:23,940 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:455350x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:23,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45535-0x10168d63052000a connected 2023-07-15 11:55:23,942 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:23,942 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:23,943 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-15 11:55:23,943 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45535 2023-07-15 11:55:23,943 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45535 2023-07-15 11:55:23,944 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45535 2023-07-15 11:55:23,944 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45535 2023-07-15 11:55:23,944 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45535 2023-07-15 11:55:23,958 INFO [Listener at localhost/46025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-15 11:55:23,958 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,958 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,958 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:23,959 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:23,959 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:23,959 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:23,961 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43589 2023-07-15 11:55:23,961 INFO [Listener at localhost/46025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-15 11:55:23,962 DEBUG [Listener at localhost/46025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-15 11:55:23,962 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,964 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:23,965 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43589 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:23,969 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:435890x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:23,970 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:435890x0, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:23,971 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43589-0x10168d63052000b connected 2023-07-15 11:55:23,971 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:23,972 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-15 11:55:23,972 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43589 2023-07-15 11:55:23,972 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43589 2023-07-15 11:55:23,973 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43589 2023-07-15 11:55:23,973 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43589 2023-07-15 11:55:23,973 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43589 2023-07-15 11:55:23,975 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:23,976 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-15 11:55:23,977 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:23,978 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-15 11:55:23,979 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-15 11:55:23,979 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-15 11:55:23,979 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:23,981 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-15 11:55:23,981 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-15 11:55:23,981 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-15 11:55:23,983 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-15 11:55:23,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,36081,1689422123834 from backup master directory 2023-07-15 11:55:23,987 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:23,987 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-15 11:55:23,987 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:23,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:24,014 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/hbase.id with ID: e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:24,031 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:24,034 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:24,055 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7b6c49dd to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:24,058 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c113258, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:24,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:24,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-15 11:55:24,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:24,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:24,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store-tmp 2023-07-15 11:55:24,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:24,082 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-15 11:55:24,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:24,082 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:24,082 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-15 11:55:24,082 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:24,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:24,082 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:24,083 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:24,085 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:24,087 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36081%2C1689422123834.jenkins-hbase4.apache.org%2C36081%2C1689422123834.regiongroup-0, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,36081,1689422123834, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/oldWALs, maxLogs=10 2023-07-15 11:55:24,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,36081,1689422123834/jenkins-hbase4.apache.org%2C36081%2C1689422123834.jenkins-hbase4.apache.org%2C36081%2C1689422123834.regiongroup-0.1689422124088 2023-07-15 11:55:24,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-274df9f8-c308-48fc-8592-c230a3934cdd,DISK], DatanodeInfoWithStorage[127.0.0.1:45131,DS-caa7d3ae-4d76-4779-b495-05a972fbb935,DISK]] 2023-07-15 11:55:24,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:24,104 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:24,104 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:24,104 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:24,107 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:24,110 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-15 11:55:24,110 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-15 11:55:24,111 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:24,113 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:24,114 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:24,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:24,125 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-15 11:55:24,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64078264, jitterRate=-0.04515945911407471}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-15 11:55:24,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:24,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-15 11:55:24,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-15 11:55:24,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-15 11:55:24,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-15 11:55:24,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-15 11:55:24,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-07-15 11:55:24,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-15 11:55:24,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-15 11:55:24,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-15 11:55:24,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-15 11:55:24,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-15 11:55:24,157 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-15 11:55:24,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-15 11:55:24,157 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-15 11:55:24,160 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:24,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-15 11:55:24,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-15 11:55:24,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-15 11:55:24,166 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-15 11:55:24,166 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-15 11:55:24,166 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-15 11:55:24,166 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-15 11:55:24,166 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-15 11:55:24,166 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:24,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,36081,1689422123834, sessionid=0x10168d630520007, setting cluster-up flag (Was=false) 2023-07-15 11:55:24,173 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:24,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-15 11:55:24,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:24,188 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:24,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-15 11:55:24,201 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:24,202 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.hbase-snapshot/.tmp 2023-07-15 11:55:24,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-15 11:55:24,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:24,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:24,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:24,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:24,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-15 11:55:24,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:24,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689422154213 2023-07-15 11:55:24,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-15 11:55:24,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-15 11:55:24,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-15 11:55:24,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-15 11:55:24,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-15 11:55:24,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-15 11:55:24,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-15 11:55:24,215 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-15 11:55:24,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-15 11:55:24,215 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-15 11:55:24,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-15 11:55:24,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-15 11:55:24,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-15 11:55:24,216 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422124216,5,FailOnTimeoutGroup] 2023-07-15 11:55:24,217 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:24,218 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422124216,5,FailOnTimeoutGroup] 2023-07-15 11:55:24,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-15 11:55:24,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,254 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-15 11:55:24,255 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-15 11:55:24,255 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346 2023-07-15 11:55:24,283 INFO [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(951): ClusterId : e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:24,285 DEBUG [RS:1;jenkins-hbase4:46827] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-15 11:55:24,285 INFO [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(951): ClusterId : e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:24,286 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:24,288 INFO [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(951): ClusterId : e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:24,290 DEBUG [RS:1;jenkins-hbase4:46827] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-15 11:55:24,290 DEBUG [RS:1;jenkins-hbase4:46827] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-15 11:55:24,290 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-15 11:55:24,286 INFO [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(951): ClusterId : e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:24,293 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info 2023-07-15 11:55:24,293 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-15 11:55:24,294 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:24,294 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-15 11:55:24,296 DEBUG [RS:2;jenkins-hbase4:45535] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-15 11:55:24,298 DEBUG [RS:0;jenkins-hbase4:39401] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-15 11:55:24,298 DEBUG [RS:3;jenkins-hbase4:43589] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-15 11:55:24,298 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:24,299 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-15 11:55:24,300 DEBUG [RS:1;jenkins-hbase4:46827] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-15 11:55:24,301 DEBUG [RS:0;jenkins-hbase4:39401] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-15 11:55:24,306 DEBUG [RS:2;jenkins-hbase4:45535] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-15 11:55:24,305 DEBUG [RS:1;jenkins-hbase4:46827] zookeeper.ReadOnlyZKClient(139): Connect 0x3222ebea to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:24,301 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:24,306 DEBUG [RS:2;jenkins-hbase4:45535] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-15 11:55:24,306 DEBUG [RS:0;jenkins-hbase4:39401] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-15 11:55:24,307 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-15 11:55:24,309 DEBUG [RS:3;jenkins-hbase4:43589] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-15 11:55:24,309 DEBUG [RS:3;jenkins-hbase4:43589] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-15 11:55:24,310 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table 2023-07-15 11:55:24,311 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-15 11:55:24,312 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:24,325 DEBUG [RS:3;jenkins-hbase4:43589] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-15 11:55:24,325 DEBUG [RS:2;jenkins-hbase4:45535] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-15 11:55:24,325 DEBUG [RS:0;jenkins-hbase4:39401] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-15 11:55:24,343 DEBUG [RS:3;jenkins-hbase4:43589] zookeeper.ReadOnlyZKClient(139): Connect 0x7576e072 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:24,344 DEBUG [RS:0;jenkins-hbase4:39401] zookeeper.ReadOnlyZKClient(139): Connect 0x15836c6a to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:24,352 DEBUG [RS:2;jenkins-hbase4:45535] zookeeper.ReadOnlyZKClient(139): Connect 0x57b7b3e7 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:24,353 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740 2023-07-15 11:55:24,354 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740 2023-07-15 11:55:24,357 DEBUG [RS:1;jenkins-hbase4:46827] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3ce503de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:24,357 DEBUG [RS:1;jenkins-hbase4:46827] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@38da809a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:24,368 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-15 11:55:24,370 DEBUG [RS:2;jenkins-hbase4:45535] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@59704761, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:24,370 DEBUG [RS:3;jenkins-hbase4:43589] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7f8e8b02, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:24,370 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-15 11:55:24,371 DEBUG [RS:2;jenkins-hbase4:45535] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@184d4d5a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:24,370 DEBUG [RS:1;jenkins-hbase4:46827] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:46827 2023-07-15 11:55:24,371 DEBUG [RS:3;jenkins-hbase4:43589] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a8c1a97, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:24,371 INFO [RS:1;jenkins-hbase4:46827] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-15 11:55:24,371 INFO [RS:1;jenkins-hbase4:46827] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-15 11:55:24,371 DEBUG [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1022): About to register with Master. 2023-07-15 11:55:24,372 INFO [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36081,1689422123834 with isa=jenkins-hbase4.apache.org/172.31.14.131:46827, startcode=1689422123904 2023-07-15 11:55:24,372 DEBUG [RS:1;jenkins-hbase4:46827] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-15 11:55:24,377 DEBUG [RS:0;jenkins-hbase4:39401] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11bcea02, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:24,378 DEBUG [RS:0;jenkins-hbase4:39401] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@54be6651, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:24,378 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33085, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-15 11:55:24,384 DEBUG [RS:2;jenkins-hbase4:45535] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:45535 2023-07-15 11:55:24,384 INFO [RS:2;jenkins-hbase4:45535] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-15 11:55:24,384 INFO [RS:2;jenkins-hbase4:45535] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-15 11:55:24,384 DEBUG [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1022): About to register with Master. 2023-07-15 11:55:24,387 INFO [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36081,1689422123834 with isa=jenkins-hbase4.apache.org/172.31.14.131:45535, startcode=1689422123929 2023-07-15 11:55:24,387 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36081] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,387 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-15 11:55:24,387 DEBUG [RS:2;jenkins-hbase4:45535] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-15 11:55:24,388 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62887351, jitterRate=-0.0629054456949234}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-15 11:55:24,388 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-15 11:55:24,388 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-15 11:55:24,388 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-15 11:55:24,388 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-15 11:55:24,388 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-15 11:55:24,388 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-15 11:55:24,389 DEBUG [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346 2023-07-15 11:55:24,389 DEBUG [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46809 2023-07-15 11:55:24,389 DEBUG [RS:3;jenkins-hbase4:43589] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:43589 2023-07-15 11:55:24,389 DEBUG [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-15 11:55:24,389 INFO [RS:3;jenkins-hbase4:43589] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-15 11:55:24,389 INFO [RS:3;jenkins-hbase4:43589] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-15 11:55:24,389 DEBUG [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(1022): About to register with Master. 2023-07-15 11:55:24,390 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-15 11:55:24,391 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-15 11:55:24,396 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:24,398 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-15 11:55:24,403 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-15 11:55:24,397 INFO [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36081,1689422123834 with isa=jenkins-hbase4.apache.org/172.31.14.131:43589, startcode=1689422123957 2023-07-15 11:55:24,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-15 11:55:24,403 DEBUG [RS:3;jenkins-hbase4:43589] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-15 11:55:24,405 DEBUG [RS:1;jenkins-hbase4:46827] zookeeper.ZKUtil(162): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,405 WARN [RS:1;jenkins-hbase4:46827] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:24,405 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56335, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-15 11:55:24,405 INFO [RS:1;jenkins-hbase4:46827] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:24,407 INFO [RS:1;jenkins-hbase4:46827] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:24,407 DEBUG [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,408 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46827,1689422123904] 2023-07-15 11:55:24,409 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-15 11:55:24,410 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33411, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-15 11:55:24,411 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36081] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:24,412 DEBUG [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346 2023-07-15 11:55:24,412 DEBUG [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46809 2023-07-15 11:55:24,412 DEBUG [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-15 11:55:24,415 DEBUG [RS:0;jenkins-hbase4:39401] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:39401 2023-07-15 11:55:24,415 INFO [RS:0;jenkins-hbase4:39401] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-15 11:55:24,415 INFO [RS:0;jenkins-hbase4:39401] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-15 11:55:24,415 DEBUG [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1022): About to register with Master. 2023-07-15 11:55:24,417 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36081] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:24,417 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:24,417 INFO [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36081,1689422123834 with isa=jenkins-hbase4.apache.org/172.31.14.131:39401, startcode=1689422123877 2023-07-15 11:55:24,417 DEBUG [RS:2;jenkins-hbase4:45535] zookeeper.ZKUtil(162): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:24,417 DEBUG [RS:0;jenkins-hbase4:39401] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-15 11:55:24,417 WARN [RS:2;jenkins-hbase4:45535] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:24,418 INFO [RS:2;jenkins-hbase4:45535] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:24,418 INFO [RS:2;jenkins-hbase4:45535] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:24,418 DEBUG [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:24,418 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-15 11:55:24,421 DEBUG [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346 2023-07-15 11:55:24,423 DEBUG [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46809 2023-07-15 11:55:24,423 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60471, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-15 11:55:24,423 DEBUG [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-15 11:55:24,423 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36081] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:24,424 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45535,1689422123929] 2023-07-15 11:55:24,424 DEBUG [RS:1;jenkins-hbase4:46827] zookeeper.ZKUtil(162): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,425 DEBUG [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346 2023-07-15 11:55:24,425 DEBUG [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46809 2023-07-15 11:55:24,425 DEBUG [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-15 11:55:24,426 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:24,426 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:24,426 DEBUG [RS:1;jenkins-hbase4:46827] zookeeper.ZKUtil(162): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:24,427 DEBUG [RS:3;jenkins-hbase4:43589] zookeeper.ZKUtil(162): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:24,427 WARN [RS:3;jenkins-hbase4:43589] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:24,427 INFO [RS:3;jenkins-hbase4:43589] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:24,427 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43589,1689422123957] 2023-07-15 11:55:24,427 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39401,1689422123877] 2023-07-15 11:55:24,427 INFO [RS:3;jenkins-hbase4:43589] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:24,427 DEBUG [RS:0;jenkins-hbase4:39401] zookeeper.ZKUtil(162): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:24,428 DEBUG [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:24,428 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,428 DEBUG [RS:2;jenkins-hbase4:45535] zookeeper.ZKUtil(162): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,428 WARN [RS:0;jenkins-hbase4:39401] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:24,428 INFO [RS:0;jenkins-hbase4:39401] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:24,428 DEBUG [RS:2;jenkins-hbase4:45535] zookeeper.ZKUtil(162): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:24,428 INFO [RS:0;jenkins-hbase4:39401] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:24,428 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:24,429 DEBUG [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:24,428 DEBUG [RS:1;jenkins-hbase4:46827] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:55:24,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:24,430 DEBUG [RS:2;jenkins-hbase4:45535] zookeeper.ZKUtil(162): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:24,430 INFO [RS:1;jenkins-hbase4:46827] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-15 11:55:24,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:24,432 DEBUG [RS:2;jenkins-hbase4:45535] zookeeper.ZKUtil(162): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:24,436 DEBUG [RS:2;jenkins-hbase4:45535] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:55:24,437 INFO [RS:2;jenkins-hbase4:45535] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-15 11:55:24,439 INFO [RS:1;jenkins-hbase4:46827] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-15 11:55:24,441 DEBUG [RS:0;jenkins-hbase4:39401] zookeeper.ZKUtil(162): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,441 DEBUG [RS:3;jenkins-hbase4:43589] zookeeper.ZKUtil(162): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,441 DEBUG [RS:0;jenkins-hbase4:39401] zookeeper.ZKUtil(162): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:24,441 DEBUG [RS:3;jenkins-hbase4:43589] zookeeper.ZKUtil(162): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:24,442 DEBUG [RS:0;jenkins-hbase4:39401] zookeeper.ZKUtil(162): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:24,442 DEBUG [RS:3;jenkins-hbase4:43589] zookeeper.ZKUtil(162): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:24,442 DEBUG [RS:0;jenkins-hbase4:39401] zookeeper.ZKUtil(162): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:24,442 DEBUG [RS:3;jenkins-hbase4:43589] zookeeper.ZKUtil(162): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:24,444 INFO [RS:1;jenkins-hbase4:46827] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-15 11:55:24,444 INFO [RS:1;jenkins-hbase4:46827] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,445 DEBUG [RS:0;jenkins-hbase4:39401] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:55:24,445 DEBUG [RS:3;jenkins-hbase4:43589] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:55:24,446 INFO [RS:3;jenkins-hbase4:43589] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-15 11:55:24,446 INFO [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-15 11:55:24,448 INFO [RS:0;jenkins-hbase4:39401] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-15 11:55:24,449 INFO [RS:2;jenkins-hbase4:45535] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-15 11:55:24,449 INFO [RS:3;jenkins-hbase4:43589] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-15 11:55:24,449 INFO [RS:3;jenkins-hbase4:43589] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-15 11:55:24,449 INFO [RS:3;jenkins-hbase4:43589] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,451 INFO [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-15 11:55:24,455 INFO [RS:0;jenkins-hbase4:39401] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-15 11:55:24,455 INFO [RS:2;jenkins-hbase4:45535] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-15 11:55:24,455 INFO [RS:2;jenkins-hbase4:45535] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,459 INFO [RS:1;jenkins-hbase4:46827] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,459 INFO [RS:0;jenkins-hbase4:39401] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-15 11:55:24,459 DEBUG [RS:1;jenkins-hbase4:46827] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,459 INFO [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-15 11:55:24,459 INFO [RS:3;jenkins-hbase4:43589] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,459 DEBUG [RS:1;jenkins-hbase4:46827] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,459 INFO [RS:0;jenkins-hbase4:39401] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,460 DEBUG [RS:1;jenkins-hbase4:46827] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,460 DEBUG [RS:3;jenkins-hbase4:43589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,460 DEBUG [RS:1;jenkins-hbase4:46827] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,460 DEBUG [RS:3;jenkins-hbase4:43589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,461 DEBUG [RS:3;jenkins-hbase4:43589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,461 DEBUG [RS:3;jenkins-hbase4:43589] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,461 DEBUG [RS:3;jenkins-hbase4:43589] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,461 DEBUG [RS:3;jenkins-hbase4:43589] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:24,461 DEBUG [RS:3;jenkins-hbase4:43589] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,461 DEBUG [RS:3;jenkins-hbase4:43589] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,461 DEBUG [RS:3;jenkins-hbase4:43589] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,461 INFO [RS:2;jenkins-hbase4:45535] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,460 DEBUG [RS:1;jenkins-hbase4:46827] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,460 INFO [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-15 11:55:24,461 DEBUG [RS:2;jenkins-hbase4:45535] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,461 DEBUG [RS:1;jenkins-hbase4:46827] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:24,461 DEBUG [RS:2;jenkins-hbase4:45535] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,461 DEBUG [RS:3;jenkins-hbase4:43589] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,462 DEBUG [RS:2;jenkins-hbase4:45535] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,462 DEBUG [RS:2;jenkins-hbase4:45535] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,462 DEBUG [RS:2;jenkins-hbase4:45535] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,462 DEBUG [RS:2;jenkins-hbase4:45535] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:24,462 DEBUG [RS:2;jenkins-hbase4:45535] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,462 DEBUG [RS:2;jenkins-hbase4:45535] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,462 DEBUG [RS:2;jenkins-hbase4:45535] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,462 DEBUG [RS:2;jenkins-hbase4:45535] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,461 DEBUG [RS:1;jenkins-hbase4:46827] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,463 DEBUG [RS:1;jenkins-hbase4:46827] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,463 DEBUG [RS:1;jenkins-hbase4:46827] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,463 DEBUG [RS:1;jenkins-hbase4:46827] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,469 INFO [RS:0;jenkins-hbase4:39401] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,470 INFO [RS:2;jenkins-hbase4:45535] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,470 INFO [RS:1;jenkins-hbase4:46827] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,471 INFO [RS:2;jenkins-hbase4:45535] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,471 INFO [RS:1;jenkins-hbase4:46827] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,471 INFO [RS:2;jenkins-hbase4:45535] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,471 INFO [RS:1;jenkins-hbase4:46827] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,474 INFO [RS:3;jenkins-hbase4:43589] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,474 INFO [RS:3;jenkins-hbase4:43589] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,474 INFO [RS:3;jenkins-hbase4:43589] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,475 DEBUG [RS:0;jenkins-hbase4:39401] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,475 DEBUG [RS:0;jenkins-hbase4:39401] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,475 DEBUG [RS:0;jenkins-hbase4:39401] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,475 DEBUG [RS:0;jenkins-hbase4:39401] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,475 DEBUG [RS:0;jenkins-hbase4:39401] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,475 DEBUG [RS:0;jenkins-hbase4:39401] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:24,475 DEBUG [RS:0;jenkins-hbase4:39401] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,475 DEBUG [RS:0;jenkins-hbase4:39401] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,475 DEBUG [RS:0;jenkins-hbase4:39401] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,475 DEBUG [RS:0;jenkins-hbase4:39401] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:24,477 INFO [RS:0;jenkins-hbase4:39401] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,477 INFO [RS:0;jenkins-hbase4:39401] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,477 INFO [RS:0;jenkins-hbase4:39401] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,494 INFO [RS:1;jenkins-hbase4:46827] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-15 11:55:24,495 INFO [RS:1;jenkins-hbase4:46827] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46827,1689422123904-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,497 INFO [RS:2;jenkins-hbase4:45535] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-15 11:55:24,497 INFO [RS:2;jenkins-hbase4:45535] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45535,1689422123929-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,498 INFO [RS:3;jenkins-hbase4:43589] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-15 11:55:24,499 INFO [RS:3;jenkins-hbase4:43589] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43589,1689422123957-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,499 INFO [RS:0;jenkins-hbase4:39401] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-15 11:55:24,500 INFO [RS:0;jenkins-hbase4:39401] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39401,1689422123877-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,512 INFO [RS:1;jenkins-hbase4:46827] regionserver.Replication(203): jenkins-hbase4.apache.org,46827,1689422123904 started 2023-07-15 11:55:24,512 INFO [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46827,1689422123904, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46827, sessionid=0x10168d630520009 2023-07-15 11:55:24,512 INFO [RS:2;jenkins-hbase4:45535] regionserver.Replication(203): jenkins-hbase4.apache.org,45535,1689422123929 started 2023-07-15 11:55:24,512 DEBUG [RS:1;jenkins-hbase4:46827] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-15 11:55:24,512 DEBUG [RS:1;jenkins-hbase4:46827] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,512 INFO [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45535,1689422123929, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45535, sessionid=0x10168d63052000a 2023-07-15 11:55:24,512 DEBUG [RS:1;jenkins-hbase4:46827] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46827,1689422123904' 2023-07-15 11:55:24,512 DEBUG [RS:2;jenkins-hbase4:45535] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-15 11:55:24,513 DEBUG [RS:2;jenkins-hbase4:45535] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:24,513 DEBUG [RS:2;jenkins-hbase4:45535] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45535,1689422123929' 2023-07-15 11:55:24,513 DEBUG [RS:2;jenkins-hbase4:45535] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-15 11:55:24,512 DEBUG [RS:1;jenkins-hbase4:46827] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-15 11:55:24,513 DEBUG [RS:1;jenkins-hbase4:46827] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-15 11:55:24,513 DEBUG [RS:2;jenkins-hbase4:45535] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-15 11:55:24,516 DEBUG [RS:1;jenkins-hbase4:46827] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-15 11:55:24,516 DEBUG [RS:2;jenkins-hbase4:45535] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-15 11:55:24,516 DEBUG [RS:2;jenkins-hbase4:45535] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-15 11:55:24,516 DEBUG [RS:1;jenkins-hbase4:46827] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-15 11:55:24,516 DEBUG [RS:1;jenkins-hbase4:46827] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,516 DEBUG [RS:1;jenkins-hbase4:46827] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46827,1689422123904' 2023-07-15 11:55:24,516 DEBUG [RS:1;jenkins-hbase4:46827] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-15 11:55:24,516 DEBUG [RS:2;jenkins-hbase4:45535] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:24,516 DEBUG [RS:2;jenkins-hbase4:45535] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45535,1689422123929' 2023-07-15 11:55:24,516 DEBUG [RS:2;jenkins-hbase4:45535] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-15 11:55:24,516 DEBUG [RS:1;jenkins-hbase4:46827] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-15 11:55:24,516 DEBUG [RS:2;jenkins-hbase4:45535] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-15 11:55:24,517 DEBUG [RS:1;jenkins-hbase4:46827] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-15 11:55:24,517 INFO [RS:1;jenkins-hbase4:46827] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-15 11:55:24,517 INFO [RS:1;jenkins-hbase4:46827] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-15 11:55:24,517 DEBUG [RS:2;jenkins-hbase4:45535] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-15 11:55:24,517 INFO [RS:2;jenkins-hbase4:45535] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-15 11:55:24,517 INFO [RS:2;jenkins-hbase4:45535] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-15 11:55:24,517 INFO [RS:3;jenkins-hbase4:43589] regionserver.Replication(203): jenkins-hbase4.apache.org,43589,1689422123957 started 2023-07-15 11:55:24,517 INFO [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43589,1689422123957, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43589, sessionid=0x10168d63052000b 2023-07-15 11:55:24,517 INFO [RS:0;jenkins-hbase4:39401] regionserver.Replication(203): jenkins-hbase4.apache.org,39401,1689422123877 started 2023-07-15 11:55:24,517 DEBUG [RS:3;jenkins-hbase4:43589] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-15 11:55:24,518 INFO [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,39401,1689422123877, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39401, sessionid=0x10168d630520008 2023-07-15 11:55:24,518 DEBUG [RS:3;jenkins-hbase4:43589] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:24,518 DEBUG [RS:0;jenkins-hbase4:39401] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-15 11:55:24,518 DEBUG [RS:0;jenkins-hbase4:39401] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:24,518 DEBUG [RS:0;jenkins-hbase4:39401] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39401,1689422123877' 2023-07-15 11:55:24,518 DEBUG [RS:0;jenkins-hbase4:39401] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-15 11:55:24,518 DEBUG [RS:3;jenkins-hbase4:43589] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43589,1689422123957' 2023-07-15 11:55:24,518 DEBUG [RS:3;jenkins-hbase4:43589] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-15 11:55:24,518 DEBUG [RS:0;jenkins-hbase4:39401] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-15 11:55:24,518 DEBUG [RS:3;jenkins-hbase4:43589] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-15 11:55:24,519 DEBUG [RS:0;jenkins-hbase4:39401] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-15 11:55:24,519 DEBUG [RS:3;jenkins-hbase4:43589] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-15 11:55:24,519 DEBUG [RS:0;jenkins-hbase4:39401] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-15 11:55:24,519 DEBUG [RS:0;jenkins-hbase4:39401] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:24,519 DEBUG [RS:3;jenkins-hbase4:43589] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-15 11:55:24,519 DEBUG [RS:0;jenkins-hbase4:39401] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39401,1689422123877' 2023-07-15 11:55:24,519 DEBUG [RS:0;jenkins-hbase4:39401] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-15 11:55:24,519 DEBUG [RS:3;jenkins-hbase4:43589] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:24,519 DEBUG [RS:3;jenkins-hbase4:43589] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43589,1689422123957' 2023-07-15 11:55:24,519 DEBUG [RS:3;jenkins-hbase4:43589] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-15 11:55:24,520 DEBUG [RS:3;jenkins-hbase4:43589] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-15 11:55:24,520 DEBUG [RS:0;jenkins-hbase4:39401] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-15 11:55:24,520 DEBUG [RS:3;jenkins-hbase4:43589] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-15 11:55:24,520 INFO [RS:3;jenkins-hbase4:43589] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-15 11:55:24,520 INFO [RS:3;jenkins-hbase4:43589] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-15 11:55:24,520 DEBUG [RS:0;jenkins-hbase4:39401] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-15 11:55:24,521 INFO [RS:0;jenkins-hbase4:39401] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-15 11:55:24,521 INFO [RS:0;jenkins-hbase4:39401] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-15 11:55:24,568 DEBUG [jenkins-hbase4:36081] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-07-15 11:55:24,569 DEBUG [jenkins-hbase4:36081] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-15 11:55:24,571 DEBUG [jenkins-hbase4:36081] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-15 11:55:24,571 DEBUG [jenkins-hbase4:36081] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-15 11:55:24,571 DEBUG [jenkins-hbase4:36081] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-15 11:55:24,571 DEBUG [jenkins-hbase4:36081] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-15 11:55:24,571 DEBUG [jenkins-hbase4:36081] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-15 11:55:24,573 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46827,1689422123904, state=OPENING 2023-07-15 11:55:24,575 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-15 11:55:24,576 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:24,576 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46827,1689422123904}] 2023-07-15 11:55:24,576 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-15 11:55:24,617 INFO [RS:1;jenkins-hbase4:46827] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:24,617 INFO [RS:2;jenkins-hbase4:45535] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:24,620 INFO [RS:2;jenkins-hbase4:45535] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45535%2C1689422123929.jenkins-hbase4.apache.org%2C45535%2C1689422123929.regiongroup-0, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:24,620 INFO [RS:1;jenkins-hbase4:46827] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46827%2C1689422123904.jenkins-hbase4.apache.org%2C46827%2C1689422123904.regiongroup-0, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:24,620 INFO [RS:3;jenkins-hbase4:43589] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:24,622 INFO [RS:0;jenkins-hbase4:39401] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:24,625 INFO [RS:3;jenkins-hbase4:43589] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43589%2C1689422123957.jenkins-hbase4.apache.org%2C43589%2C1689422123957.regiongroup-0, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,43589,1689422123957, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:24,626 INFO [RS:0;jenkins-hbase4:39401] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39401%2C1689422123877.jenkins-hbase4.apache.org%2C39401%2C1689422123877.regiongroup-0, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:24,695 INFO [RS:2;jenkins-hbase4:45535] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929/jenkins-hbase4.apache.org%2C45535%2C1689422123929.jenkins-hbase4.apache.org%2C45535%2C1689422123929.regiongroup-0.1689422124625 2023-07-15 11:55:24,706 DEBUG [RS:2;jenkins-hbase4:45535] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:45131,DS-caa7d3ae-4d76-4779-b495-05a972fbb935,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK]] 2023-07-15 11:55:24,715 INFO [RS:1;jenkins-hbase4:46827] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904/jenkins-hbase4.apache.org%2C46827%2C1689422123904.jenkins-hbase4.apache.org%2C46827%2C1689422123904.regiongroup-0.1689422124625 2023-07-15 11:55:24,716 DEBUG [RS:1;jenkins-hbase4:46827] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:45131,DS-caa7d3ae-4d76-4779-b495-05a972fbb935,DISK]] 2023-07-15 11:55:24,717 INFO [RS:3;jenkins-hbase4:43589] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,43589,1689422123957/jenkins-hbase4.apache.org%2C43589%2C1689422123957.jenkins-hbase4.apache.org%2C43589%2C1689422123957.regiongroup-0.1689422124643 2023-07-15 11:55:24,717 DEBUG [RS:3;jenkins-hbase4:43589] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45131,DS-caa7d3ae-4d76-4779-b495-05a972fbb935,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-274df9f8-c308-48fc-8592-c230a3934cdd,DISK]] 2023-07-15 11:55:24,719 INFO [RS:0;jenkins-hbase4:39401] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877/jenkins-hbase4.apache.org%2C39401%2C1689422123877.jenkins-hbase4.apache.org%2C39401%2C1689422123877.regiongroup-0.1689422124634 2023-07-15 11:55:24,719 DEBUG [RS:0;jenkins-hbase4:39401] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33417,DS-274df9f8-c308-48fc-8592-c230a3934cdd,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK]] 2023-07-15 11:55:24,733 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:24,733 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:24,736 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47884, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-15 11:55:24,743 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-15 11:55:24,743 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:24,743 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:24,743 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:24,745 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46827%2C1689422123904.meta, suffix=.meta, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:24,761 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904/jenkins-hbase4.apache.org%2C46827%2C1689422123904.meta.1689422124747.meta 2023-07-15 11:55:24,761 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:45131,DS-caa7d3ae-4d76-4779-b495-05a972fbb935,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK]] 2023-07-15 11:55:24,761 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:24,762 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-15 11:55:24,762 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-15 11:55:24,762 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-15 11:55:24,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-15 11:55:24,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:24,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-15 11:55:24,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-15 11:55:24,767 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-15 11:55:24,768 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info 2023-07-15 11:55:24,768 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info 2023-07-15 11:55:24,768 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-15 11:55:24,769 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:24,769 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-15 11:55:24,770 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:24,770 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:24,771 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-15 11:55:24,771 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:24,772 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-15 11:55:24,772 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table 2023-07-15 11:55:24,773 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table 2023-07-15 11:55:24,773 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-15 11:55:24,773 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:24,775 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740 2023-07-15 11:55:24,777 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740 2023-07-15 11:55:24,780 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-15 11:55:24,782 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-15 11:55:24,783 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73157066, jitterRate=0.09012523293495178}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-15 11:55:24,783 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-15 11:55:24,786 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689422124733 2023-07-15 11:55:24,792 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-15 11:55:24,793 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-15 11:55:24,794 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46827,1689422123904, state=OPEN 2023-07-15 11:55:24,796 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-15 11:55:24,796 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-15 11:55:24,800 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-15 11:55:24,800 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46827,1689422123904 in 220 msec 2023-07-15 11:55:24,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-15 11:55:24,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 397 msec 2023-07-15 11:55:24,808 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 602 msec 2023-07-15 11:55:24,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689422124808, completionTime=-1 2023-07-15 11:55:24,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-07-15 11:55:24,808 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-15 11:55:24,813 DEBUG [hconnection-0x103135dd-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:24,817 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47898, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:24,819 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-07-15 11:55:24,819 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689422184819 2023-07-15 11:55:24,819 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689422244819 2023-07-15 11:55:24,820 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 11 msec 2023-07-15 11:55:24,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36081,1689422123834-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36081,1689422123834-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36081,1689422123834-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:36081, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:24,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-15 11:55:24,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:24,832 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-15 11:55:24,832 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-15 11:55:24,834 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-15 11:55:24,835 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-15 11:55:24,840 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:24,841 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734 empty. 2023-07-15 11:55:24,841 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:24,841 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-15 11:55:24,861 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-15 11:55:24,863 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => faf8bf9c0e6d77a0bdf0f3a70ab43734, NAME => 'hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp 2023-07-15 11:55:24,883 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:24,883 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing faf8bf9c0e6d77a0bdf0f3a70ab43734, disabling compactions & flushes 2023-07-15 11:55:24,884 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:24,884 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:24,884 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. after waiting 0 ms 2023-07-15 11:55:24,884 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:24,884 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:24,884 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for faf8bf9c0e6d77a0bdf0f3a70ab43734: 2023-07-15 11:55:24,888 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-15 11:55:24,890 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422124890"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689422124890"}]},"ts":"1689422124890"} 2023-07-15 11:55:24,895 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-15 11:55:24,896 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-15 11:55:24,897 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422124896"}]},"ts":"1689422124896"} 2023-07-15 11:55:24,899 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-15 11:55:24,904 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-15 11:55:24,905 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-15 11:55:24,905 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-15 11:55:24,905 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-15 11:55:24,905 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-15 11:55:24,905 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-15 11:55:24,906 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN}] 2023-07-15 11:55:24,909 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN 2023-07-15 11:55:24,911 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39401,1689422123877; forceNewPlan=false, retain=false 2023-07-15 11:55:25,061 INFO [jenkins-hbase4:36081] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-15 11:55:25,063 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=faf8bf9c0e6d77a0bdf0f3a70ab43734, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:25,063 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422125063"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422125063"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422125063"}]},"ts":"1689422125063"} 2023-07-15 11:55:25,068 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure faf8bf9c0e6d77a0bdf0f3a70ab43734, server=jenkins-hbase4.apache.org,39401,1689422123877}] 2023-07-15 11:55:25,223 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:25,224 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:25,229 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54354, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-15 11:55:25,235 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:25,235 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:25,237 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39401%2C1689422123877.jenkins-hbase4.apache.org%2C39401%2C1689422123877.regiongroup-1, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:25,261 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877/jenkins-hbase4.apache.org%2C39401%2C1689422123877.jenkins-hbase4.apache.org%2C39401%2C1689422123877.regiongroup-1.1689422125238 2023-07-15 11:55:25,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45131,DS-caa7d3ae-4d76-4779-b495-05a972fbb935,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK]] 2023-07-15 11:55:25,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => faf8bf9c0e6d77a0bdf0f3a70ab43734, NAME => 'hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:25,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:25,262 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:25,262 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:25,262 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:25,264 INFO [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:25,267 DEBUG [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/info 2023-07-15 11:55:25,267 DEBUG [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/info 2023-07-15 11:55:25,268 INFO [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region faf8bf9c0e6d77a0bdf0f3a70ab43734 columnFamilyName info 2023-07-15 11:55:25,270 INFO [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] regionserver.HStore(310): Store=faf8bf9c0e6d77a0bdf0f3a70ab43734/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:25,272 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:25,273 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:25,277 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:25,283 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-15 11:55:25,284 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened faf8bf9c0e6d77a0bdf0f3a70ab43734; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74111831, jitterRate=0.10435234010219574}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-15 11:55:25,284 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for faf8bf9c0e6d77a0bdf0f3a70ab43734: 2023-07-15 11:55:25,287 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734., pid=6, masterSystemTime=1689422125223 2023-07-15 11:55:25,293 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:25,294 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:25,295 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=faf8bf9c0e6d77a0bdf0f3a70ab43734, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:25,296 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422125295"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422125295"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422125295"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422125295"}]},"ts":"1689422125295"} 2023-07-15 11:55:25,305 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-15 11:55:25,305 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure faf8bf9c0e6d77a0bdf0f3a70ab43734, server=jenkins-hbase4.apache.org,39401,1689422123877 in 231 msec 2023-07-15 11:55:25,310 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-15 11:55:25,311 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN in 399 msec 2023-07-15 11:55:25,312 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-15 11:55:25,313 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422125312"}]},"ts":"1689422125312"} 2023-07-15 11:55:25,315 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-15 11:55:25,319 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-15 11:55:25,322 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 489 msec 2023-07-15 11:55:25,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-07-15 11:55:25,343 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-07-15 11:55:25,343 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:25,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:25,351 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54358, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:25,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-15 11:55:25,368 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-15 11:55:25,374 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 19 msec 2023-07-15 11:55:25,377 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-15 11:55:25,390 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-15 11:55:25,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-07-15 11:55:25,405 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-15 11:55:25,407 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-15 11:55:25,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.420sec 2023-07-15 11:55:25,408 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-15 11:55:25,408 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-15 11:55:25,408 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-15 11:55:25,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36081,1689422123834-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-15 11:55:25,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36081,1689422123834-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-15 11:55:25,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-15 11:55:25,484 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(139): Connect 0x1e6db056 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:25,491 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@71db1572, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:25,493 DEBUG [hconnection-0x24a045a8-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:25,496 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47914, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:25,498 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:25,508 DEBUG [Listener at localhost/46025] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-15 11:55:25,511 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50922, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-15 11:55:25,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:25,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-15 11:55:25,531 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-15 11:55:25,534 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-15 11:55:25,536 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-15 11:55:25,537 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:25,537 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3 empty. 2023-07-15 11:55:25,538 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:25,538 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-15 11:55:25,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-15 11:55:25,564 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-15 11:55:25,565 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => eee02c0767aa2be4d83f125ea78097e3, NAME => 't1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp 2023-07-15 11:55:25,579 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:25,579 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing eee02c0767aa2be4d83f125ea78097e3, disabling compactions & flushes 2023-07-15 11:55:25,579 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:25,579 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:25,579 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. after waiting 0 ms 2023-07-15 11:55:25,579 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:25,579 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:25,579 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for eee02c0767aa2be4d83f125ea78097e3: 2023-07-15 11:55:25,583 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-15 11:55:25,585 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422125585"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689422125585"}]},"ts":"1689422125585"} 2023-07-15 11:55:25,588 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-15 11:55:25,589 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-15 11:55:25,590 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422125589"}]},"ts":"1689422125589"} 2023-07-15 11:55:25,592 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-15 11:55:25,597 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-15 11:55:25,598 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-15 11:55:25,598 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-15 11:55:25,598 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-15 11:55:25,599 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=eee02c0767aa2be4d83f125ea78097e3, ASSIGN}] 2023-07-15 11:55:25,601 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=eee02c0767aa2be4d83f125ea78097e3, ASSIGN 2023-07-15 11:55:25,602 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=eee02c0767aa2be4d83f125ea78097e3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34419,1689422119824; forceNewPlan=false, retain=false 2023-07-15 11:55:25,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-15 11:55:25,753 INFO [jenkins-hbase4:38223] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-15 11:55:25,755 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=eee02c0767aa2be4d83f125ea78097e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:25,755 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422125754"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422125754"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422125754"}]},"ts":"1689422125754"} 2023-07-15 11:55:25,758 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure eee02c0767aa2be4d83f125ea78097e3, server=jenkins-hbase4.apache.org,34419,1689422119824}] 2023-07-15 11:55:25,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-15 11:55:25,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:25,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eee02c0767aa2be4d83f125ea78097e3, NAME => 't1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:25,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:25,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:25,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:25,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:25,923 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:25,925 DEBUG [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/cf1 2023-07-15 11:55:25,925 DEBUG [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/cf1 2023-07-15 11:55:25,925 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eee02c0767aa2be4d83f125ea78097e3 columnFamilyName cf1 2023-07-15 11:55:25,926 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] regionserver.HStore(310): Store=eee02c0767aa2be4d83f125ea78097e3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:25,926 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:25,928 DEBUG [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/norep 2023-07-15 11:55:25,928 DEBUG [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/norep 2023-07-15 11:55:25,929 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eee02c0767aa2be4d83f125ea78097e3 columnFamilyName norep 2023-07-15 11:55:25,930 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] regionserver.HStore(310): Store=eee02c0767aa2be4d83f125ea78097e3/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:25,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:25,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:25,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-15 11:55:25,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:25,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-15 11:55:25,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened eee02c0767aa2be4d83f125ea78097e3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61444105, jitterRate=-0.08441148698329926}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-15 11:55:25,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for eee02c0767aa2be4d83f125ea78097e3: 2023-07-15 11:55:25,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3., pid=11, masterSystemTime=1689422125911 2023-07-15 11:55:25,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:25,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:25,950 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=eee02c0767aa2be4d83f125ea78097e3, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:25,950 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422125950"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422125950"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422125950"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422125950"}]},"ts":"1689422125950"} 2023-07-15 11:55:25,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-15 11:55:25,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure eee02c0767aa2be4d83f125ea78097e3, server=jenkins-hbase4.apache.org,34419,1689422119824 in 196 msec 2023-07-15 11:55:25,960 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-15 11:55:25,962 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=eee02c0767aa2be4d83f125ea78097e3, ASSIGN in 358 msec 2023-07-15 11:55:25,964 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-15 11:55:25,964 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422125964"}]},"ts":"1689422125964"} 2023-07-15 11:55:25,967 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-15 11:55:25,970 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-15 11:55:25,973 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 444 msec 2023-07-15 11:55:26,156 INFO [jenkins-hbase4:35805Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:55:26,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-15 11:55:26,157 INFO [Listener at localhost/46025] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-15 11:55:26,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:26,159 INFO [jenkins-hbase4:34419Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:55:26,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-15 11:55:26,164 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-15 11:55:26,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-15 11:55:26,166 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-15 11:55:26,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-15 11:55:26,168 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:26,169 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf empty. 2023-07-15 11:55:26,169 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:26,169 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-15 11:55:26,190 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-15 11:55:26,191 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 89b547022cb2e9610cb6246270f3cbaf, NAME => 't2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.tmp 2023-07-15 11:55:26,201 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:26,202 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 89b547022cb2e9610cb6246270f3cbaf, disabling compactions & flushes 2023-07-15 11:55:26,202 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:26,202 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:26,202 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. after waiting 0 ms 2023-07-15 11:55:26,202 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:26,202 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:26,202 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 89b547022cb2e9610cb6246270f3cbaf: 2023-07-15 11:55:26,206 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-15 11:55:26,208 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422126207"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689422126207"}]},"ts":"1689422126207"} 2023-07-15 11:55:26,210 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-15 11:55:26,211 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-15 11:55:26,211 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422126211"}]},"ts":"1689422126211"} 2023-07-15 11:55:26,213 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-15 11:55:26,219 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-15 11:55:26,221 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-15 11:55:26,221 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-15 11:55:26,221 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-15 11:55:26,221 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=89b547022cb2e9610cb6246270f3cbaf, ASSIGN}] 2023-07-15 11:55:26,223 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=89b547022cb2e9610cb6246270f3cbaf, ASSIGN 2023-07-15 11:55:26,225 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=89b547022cb2e9610cb6246270f3cbaf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34419,1689422119824; forceNewPlan=false, retain=false 2023-07-15 11:55:26,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-15 11:55:26,376 INFO [jenkins-hbase4:38223] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-15 11:55:26,378 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=89b547022cb2e9610cb6246270f3cbaf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:26,378 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422126378"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422126378"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422126378"}]},"ts":"1689422126378"} 2023-07-15 11:55:26,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 89b547022cb2e9610cb6246270f3cbaf, server=jenkins-hbase4.apache.org,34419,1689422119824}] 2023-07-15 11:55:26,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-15 11:55:26,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:26,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 89b547022cb2e9610cb6246270f3cbaf, NAME => 't2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:26,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:26,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:26,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:26,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:26,564 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:26,568 DEBUG [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/cf1 2023-07-15 11:55:26,568 DEBUG [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/cf1 2023-07-15 11:55:26,568 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 89b547022cb2e9610cb6246270f3cbaf columnFamilyName cf1 2023-07-15 11:55:26,569 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] regionserver.HStore(310): Store=89b547022cb2e9610cb6246270f3cbaf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:26,569 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:26,572 DEBUG [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/norep 2023-07-15 11:55:26,572 DEBUG [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/norep 2023-07-15 11:55:26,573 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 89b547022cb2e9610cb6246270f3cbaf columnFamilyName norep 2023-07-15 11:55:26,573 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] regionserver.HStore(310): Store=89b547022cb2e9610cb6246270f3cbaf/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:26,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:26,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:26,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-15 11:55:26,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:26,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-15 11:55:26,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 89b547022cb2e9610cb6246270f3cbaf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70945556, jitterRate=0.05717116594314575}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-15 11:55:26,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 89b547022cb2e9610cb6246270f3cbaf: 2023-07-15 11:55:26,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf., pid=14, masterSystemTime=1689422126539 2023-07-15 11:55:26,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:26,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:26,592 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=89b547022cb2e9610cb6246270f3cbaf, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:26,592 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422126592"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422126592"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422126592"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422126592"}]},"ts":"1689422126592"} 2023-07-15 11:55:26,597 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-15 11:55:26,597 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 89b547022cb2e9610cb6246270f3cbaf, server=jenkins-hbase4.apache.org,34419,1689422119824 in 214 msec 2023-07-15 11:55:26,600 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-15 11:55:26,601 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=89b547022cb2e9610cb6246270f3cbaf, ASSIGN in 376 msec 2023-07-15 11:55:26,602 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-15 11:55:26,603 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422126602"}]},"ts":"1689422126602"} 2023-07-15 11:55:26,605 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-15 11:55:26,608 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-15 11:55:26,610 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 450 msec 2023-07-15 11:55:26,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-15 11:55:26,772 INFO [Listener at localhost/46025] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-15 11:55:26,776 DEBUG [Listener at localhost/46025] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-15 11:55:26,784 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56698, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-15 11:55:26,787 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:26,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-15 11:55:26,792 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-15 11:55:26,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-15 11:55:26,796 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-15 11:55:26,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-15 11:55:26,798 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:26,798 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f empty. 2023-07-15 11:55:26,799 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:26,799 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-15 11:55:26,819 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-15 11:55:26,821 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => ed49ae2049e592f2f5d8017bfcee4d2f, NAME => 't1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp 2023-07-15 11:55:26,834 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:26,834 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing ed49ae2049e592f2f5d8017bfcee4d2f, disabling compactions & flushes 2023-07-15 11:55:26,834 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:26,834 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:26,834 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. after waiting 0 ms 2023-07-15 11:55:26,834 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:26,834 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:26,834 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for ed49ae2049e592f2f5d8017bfcee4d2f: 2023-07-15 11:55:26,841 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-15 11:55:26,842 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422126842"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689422126842"}]},"ts":"1689422126842"} 2023-07-15 11:55:26,845 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-15 11:55:26,846 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-15 11:55:26,847 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422126846"}]},"ts":"1689422126846"} 2023-07-15 11:55:26,849 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-15 11:55:26,854 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-15 11:55:26,855 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-15 11:55:26,855 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-15 11:55:26,855 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-15 11:55:26,855 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-15 11:55:26,855 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-15 11:55:26,856 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN}] 2023-07-15 11:55:26,859 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN 2023-07-15 11:55:26,860 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46827,1689422123904; forceNewPlan=false, retain=false 2023-07-15 11:55:26,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-15 11:55:27,011 INFO [jenkins-hbase4:36081] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-15 11:55:27,013 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=ed49ae2049e592f2f5d8017bfcee4d2f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:27,013 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422127012"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422127012"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422127012"}]},"ts":"1689422127012"} 2023-07-15 11:55:27,016 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure ed49ae2049e592f2f5d8017bfcee4d2f, server=jenkins-hbase4.apache.org,46827,1689422123904}] 2023-07-15 11:55:27,061 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-15 11:55:27,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-15 11:55:27,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:27,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:27,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46827%2C1689422123904.jenkins-hbase4.apache.org%2C46827%2C1689422123904.regiongroup-1, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:27,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904/jenkins-hbase4.apache.org%2C46827%2C1689422123904.jenkins-hbase4.apache.org%2C46827%2C1689422123904.regiongroup-1.1689422127181 2023-07-15 11:55:27,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK], DatanodeInfoWithStorage[127.0.0.1:45131,DS-caa7d3ae-4d76-4779-b495-05a972fbb935,DISK]] 2023-07-15 11:55:27,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ed49ae2049e592f2f5d8017bfcee4d2f, NAME => 't1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:27,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:27,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:27,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:27,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:27,238 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:27,245 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1 2023-07-15 11:55:27,245 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1 2023-07-15 11:55:27,248 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed49ae2049e592f2f5d8017bfcee4d2f columnFamilyName cf1 2023-07-15 11:55:27,249 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(310): Store=ed49ae2049e592f2f5d8017bfcee4d2f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:27,249 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:27,251 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/norep 2023-07-15 11:55:27,251 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/norep 2023-07-15 11:55:27,252 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed49ae2049e592f2f5d8017bfcee4d2f columnFamilyName norep 2023-07-15 11:55:27,252 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(310): Store=ed49ae2049e592f2f5d8017bfcee4d2f/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:27,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:27,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:27,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-15 11:55:27,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:27,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-15 11:55:27,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ed49ae2049e592f2f5d8017bfcee4d2f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75057807, jitterRate=0.11844848096370697}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-15 11:55:27,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ed49ae2049e592f2f5d8017bfcee4d2f: 2023-07-15 11:55:27,273 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-15 11:55:27,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f., pid=11, masterSystemTime=1689422127170 2023-07-15 11:55:27,275 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-15 11:55:27,276 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-15 11:55:27,276 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-15 11:55:27,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:27,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:27,283 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=ed49ae2049e592f2f5d8017bfcee4d2f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:27,283 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422127282"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422127282"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422127282"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422127282"}]},"ts":"1689422127282"} 2023-07-15 11:55:27,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-15 11:55:27,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure ed49ae2049e592f2f5d8017bfcee4d2f, server=jenkins-hbase4.apache.org,46827,1689422123904 in 270 msec 2023-07-15 11:55:27,293 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-15 11:55:27,293 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN in 434 msec 2023-07-15 11:55:27,294 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-15 11:55:27,294 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422127294"}]},"ts":"1689422127294"} 2023-07-15 11:55:27,296 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-15 11:55:27,300 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-15 11:55:27,302 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 513 msec 2023-07-15 11:55:27,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-15 11:55:27,401 INFO [Listener at localhost/46025] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-15 11:55:27,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:27,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-15 11:55:27,408 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-15 11:55:27,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-15 11:55:27,410 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-15 11:55:27,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-15 11:55:27,412 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:27,412 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1 empty. 2023-07-15 11:55:27,413 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:27,413 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-15 11:55:27,433 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-15 11:55:27,435 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8bb3eb49d5c769961a9b0f7da157f1d1, NAME => 't2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.tmp 2023-07-15 11:55:27,452 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:27,452 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 8bb3eb49d5c769961a9b0f7da157f1d1, disabling compactions & flushes 2023-07-15 11:55:27,452 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:27,453 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:27,453 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. after waiting 0 ms 2023-07-15 11:55:27,453 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:27,453 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:27,453 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 8bb3eb49d5c769961a9b0f7da157f1d1: 2023-07-15 11:55:27,456 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-15 11:55:27,457 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422127457"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689422127457"}]},"ts":"1689422127457"} 2023-07-15 11:55:27,460 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-15 11:55:27,461 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-15 11:55:27,461 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422127461"}]},"ts":"1689422127461"} 2023-07-15 11:55:27,463 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-15 11:55:27,468 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-15 11:55:27,470 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-15 11:55:27,470 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-15 11:55:27,470 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-15 11:55:27,470 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-15 11:55:27,470 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-15 11:55:27,470 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN}] 2023-07-15 11:55:27,473 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN 2023-07-15 11:55:27,475 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45535,1689422123929; forceNewPlan=false, retain=false 2023-07-15 11:55:27,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-15 11:55:27,626 INFO [jenkins-hbase4:36081] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-15 11:55:27,627 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=8bb3eb49d5c769961a9b0f7da157f1d1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:27,628 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422127627"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422127627"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422127627"}]},"ts":"1689422127627"} 2023-07-15 11:55:27,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 8bb3eb49d5c769961a9b0f7da157f1d1, server=jenkins-hbase4.apache.org,45535,1689422123929}] 2023-07-15 11:55:27,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-15 11:55:27,785 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:27,785 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:27,789 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39348, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-15 11:55:27,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:27,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:27,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45535%2C1689422123929.jenkins-hbase4.apache.org%2C45535%2C1689422123929.regiongroup-1, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:27,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929/jenkins-hbase4.apache.org%2C45535%2C1689422123929.jenkins-hbase4.apache.org%2C45535%2C1689422123929.regiongroup-1.1689422127796 2023-07-15 11:55:27,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK], DatanodeInfoWithStorage[127.0.0.1:45131,DS-caa7d3ae-4d76-4779-b495-05a972fbb935,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK]] 2023-07-15 11:55:27,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8bb3eb49d5c769961a9b0f7da157f1d1, NAME => 't2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:27,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:27,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:27,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:27,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:27,810 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:27,812 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1 2023-07-15 11:55:27,812 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1 2023-07-15 11:55:27,812 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8bb3eb49d5c769961a9b0f7da157f1d1 columnFamilyName cf1 2023-07-15 11:55:27,813 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(310): Store=8bb3eb49d5c769961a9b0f7da157f1d1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:27,813 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:27,815 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/norep 2023-07-15 11:55:27,815 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/norep 2023-07-15 11:55:27,815 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8bb3eb49d5c769961a9b0f7da157f1d1 columnFamilyName norep 2023-07-15 11:55:27,816 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(310): Store=8bb3eb49d5c769961a9b0f7da157f1d1/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:27,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:27,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:27,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-15 11:55:27,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:27,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-15 11:55:27,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8bb3eb49d5c769961a9b0f7da157f1d1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58764531, jitterRate=-0.1243402510881424}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-15 11:55:27,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8bb3eb49d5c769961a9b0f7da157f1d1: 2023-07-15 11:55:27,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1., pid=14, masterSystemTime=1689422127785 2023-07-15 11:55:27,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:27,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:27,832 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=8bb3eb49d5c769961a9b0f7da157f1d1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:27,833 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422127832"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422127832"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422127832"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422127832"}]},"ts":"1689422127832"} 2023-07-15 11:55:27,838 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-15 11:55:27,838 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 8bb3eb49d5c769961a9b0f7da157f1d1, server=jenkins-hbase4.apache.org,45535,1689422123929 in 205 msec 2023-07-15 11:55:27,841 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-15 11:55:27,841 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN in 368 msec 2023-07-15 11:55:27,842 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-15 11:55:27,842 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689422127842"}]},"ts":"1689422127842"} 2023-07-15 11:55:27,844 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-15 11:55:27,848 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-15 11:55:27,851 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 445 msec 2023-07-15 11:55:28,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36081] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-15 11:55:28,015 INFO [Listener at localhost/46025] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-15 11:55:28,015 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(139): Connect 0x2a94bf46 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:28,022 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24598a72, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:28,023 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(139): Connect 0x52264cd1 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:28,034 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5d47a75c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:28,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.HMaster(3464): Client=jenkins//172.31.14.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:57666:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-07-15 11:55:28,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-15 11:55:28,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-15 11:55:28,077 INFO [PEWorker-1] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-15 11:55:28,077 INFO [PEWorker-1] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@3a7f3fc9check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:28,082 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@3a7f3fc9check-peer-cluster-id0x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:28,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@3a7f3fc9check-peer-cluster-id-0x10168d630520014 connected 2023-07-15 11:55:28,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-15 11:55:28,188 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@3a7f3fc9check-peer-cluster-id-0x10168d630520014, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:28,196 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:28,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-07-15 11:55:28,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-15 11:55:28,375 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:28,376 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:28,376 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34419] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-07-15 11:55:28,378 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59142, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-15 11:55:28,378 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-15 11:55:28,379 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35805] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-07-15 11:55:28,379 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-15 11:55:28,391 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-15 11:55:28,391 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-15 11:55:28,411 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-15 11:55:28,411 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-15 11:55:28,444 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-07-15 11:55:28,444 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-07-15 11:55:28,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38223] master.HMaster(3641): Remote procedure done, pid=17 2023-07-15 11:55:28,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] master.HMaster(3641): Remote procedure done, pid=16 2023-07-15 11:55:28,450 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38223] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,35805,1689422119867 suceeded 2023-07-15 11:55:28,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38223] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,34419,1689422119824 suceeded 2023-07-15 11:55:28,455 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 244 msec 2023-07-15 11:55:28,456 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-07-15 11:55:28,456 INFO [PEWorker-1] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:57666:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-07-15 11:55:28,456 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 244 msec 2023-07-15 11:55:28,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 404 msec 2023-07-15 11:55:28,474 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x3ddb30aa to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:28,477 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x3dbbe3eb to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:28,489 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@506fe693, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:28,493 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:28,494 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@227366f3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:28,496 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:28,499 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:28,500 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10168d630520017 connected 2023-07-15 11:55:28,500 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:28,502 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10168d630520018 connected 2023-07-15 11:55:28,502 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=1) is replicating from cluster=f0521d31-92f9-4caa-8fd4-4b1a00343aa2 to cluster=e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:28,502 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=f0521d31-92f9-4caa-8fd4-4b1a00343aa2 to cluster=e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:28,505 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0 2023-07-15 11:55:28,505 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1 2023-07-15 11:55:28,513 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-15 11:55:28,513 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-15 11:55:28,517 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0 2023-07-15 11:55:28,517 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1 2023-07-15 11:55:28,517 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-15 11:55:28,517 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0 2023-07-15 11:55:28,518 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0 2023-07-15 11:55:28,601 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:28,607 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-15 11:55:28,607 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-15 11:55:28,607 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-15 11:55:28,607 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-15 11:55:28,607 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-15 11:55:28,607 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-15 11:55:28,607 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-15 11:55:28,607 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228, lastWalPosition=901, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-15 11:55:28,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38223] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-15 11:55:28,672 DEBUG [Listener at localhost/46025] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-07-15 11:55:28,673 DEBUG [hconnection-0x286bf07d-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:28,679 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56414, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:28,706 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-15 11:55:28,706 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-15 11:55:28,714 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:28,725 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 1519 2023-07-15 11:55:28,725 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 11 WAL entries eligible for replication 2023-07-15 11:55:28,725 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 1519 2023-07-15 11:55:28,726 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/eee02c0767aa2be4d83f125ea78097e3/4=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/5=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/6=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/7=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/8=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/9=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/10=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/11=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/12=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/13=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/14=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276, lastWalPosition=1519, nbRowKeys=11, nbHFiles=0, heapSize=2288, lastSeqIds={}, endOfFile=false,usedBufferSize=2288] 2023-07-15 11:55:28,729 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-15 11:55:28,743 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:28,750 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39364, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-15 11:55:28,753 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:55:28,754 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] zookeeper.ReadOnlyZKClient(139): Connect 0x6c2b51a8 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:28,761 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@26af185, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:28,772 DEBUG [hconnection-0x3d93b016-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:28,777 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47920, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-15 11:55:28,825 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:28,840 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-15 11:55:28,847 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:55:28,849 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 7189 2023-07-15 11:55:28,850 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 54 WAL entries eligible for replication 2023-07-15 11:55:28,850 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 7189 2023-07-15 11:55:28,853 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/eee02c0767aa2be4d83f125ea78097e3/15=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/16=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/17=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/18=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/19=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/20=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/21=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/22=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/23=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/24=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/25=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/26=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/27=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/28=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/29=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/30=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/31=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/32=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/33=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/34=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/35=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/36=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/37=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/38=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/39=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/40=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/41=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/42=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/43=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/44=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/45=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/46=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/47=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/48=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/49=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/50=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/51=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/52=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/53=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/54=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/55=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/56=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/57=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/58=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/59=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/60=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/61=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/62=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/63=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/64=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/65=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/66=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/67=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/68=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276, lastWalPosition=7189, nbRowKeys=54, nbHFiles=0, heapSize=11232, lastSeqIds={}, endOfFile=false,usedBufferSize=11232] 2023-07-15 11:55:28,858 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:28,862 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47930, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-15 11:55:28,864 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46827] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:55:28,865 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46827] zookeeper.ReadOnlyZKClient(139): Connect 0x20246b24 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:28,873 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46827] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3bf58796, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:28,875 DEBUG [hconnection-0x114e086e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:28,876 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47936, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-15 11:55:28,889 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46827] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:55:28,953 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:28,967 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 2351 2023-07-15 11:55:28,967 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 14 WAL entries eligible for replication 2023-07-15 11:55:28,967 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 2351 2023-07-15 11:55:28,967 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/89b547022cb2e9610cb6246270f3cbaf/4=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/5=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/6=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/7=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/8=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/9=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/10=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/11=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/12=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/13=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/14=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/15=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/16=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/17=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228, lastWalPosition=2351, nbRowKeys=14, nbHFiles=0, heapSize=2912, lastSeqIds={}, endOfFile=false,usedBufferSize=2912] 2023-07-15 11:55:28,969 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:55:28,990 DEBUG [hconnection-0x3d93b016-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:28,992 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39372, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-15 11:55:29,011 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:55:29,014 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:29,014 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 35 WAL entries eligible for replication 2023-07-15 11:55:29,015 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/eee02c0767aa2be4d83f125ea78097e3/69=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/70=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/71=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/72=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/73=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/74=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/75=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/76=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/77=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/78=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/79=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/80=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/81=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/82=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/83=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/84=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/85=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/86=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/87=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/88=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/89=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/90=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/91=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/92=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/93=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/94=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/95=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/96=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/97=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/98=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/99=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/100=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/101=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/102=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276, lastWalPosition=10975, nbRowKeys=35, nbHFiles=0, heapSize=7280, lastSeqIds={}, endOfFile=false,usedBufferSize=7280] 2023-07-15 11:55:29,015 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:29,021 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:55:29,027 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:55:29,069 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:29,091 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 6866 2023-07-15 11:55:29,092 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 43 WAL entries eligible for replication 2023-07-15 11:55:29,092 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/89b547022cb2e9610cb6246270f3cbaf/18=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/19=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/20=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/21=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/22=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/23=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/24=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/25=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/26=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/27=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/28=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/29=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/30=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/31=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/32=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/33=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/34=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/35=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/36=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/37=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/38=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/39=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/40=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/41=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/42=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/43=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/44=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/45=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/46=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/47=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/48=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/49=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/50=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/51=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/52=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/53=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/54=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/55=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/56=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/57=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/58=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/59=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/60=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228, lastWalPosition=6866, nbRowKeys=43, nbHFiles=0, heapSize=8944, lastSeqIds={}, endOfFile=false,usedBufferSize=8944] 2023-07-15 11:55:29,093 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 6866 2023-07-15 11:55:29,109 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46827] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:55:29,116 DEBUG [hconnection-0x114e086e-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:29,123 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39382, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-15 11:55:29,132 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46827] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:55:29,132 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:29,184 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:29,216 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 11916 2023-07-15 11:55:29,217 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 48 WAL entries eligible for replication 2023-07-15 11:55:29,217 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/89b547022cb2e9610cb6246270f3cbaf/61=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/62=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/63=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/64=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/65=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/66=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/67=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/68=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/69=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/70=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/71=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/72=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/73=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/74=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/75=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/76=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/77=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/78=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/79=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/80=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/81=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/82=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/83=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/84=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/85=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/86=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/87=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/88=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/89=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/90=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/91=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/92=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/93=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/94=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/95=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/96=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/97=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/98=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/99=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/100=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/101=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/102=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/103=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/104=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/105=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/106=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/107=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/108=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228, lastWalPosition=11916, nbRowKeys=48, nbHFiles=0, heapSize=9984, lastSeqIds={}, endOfFile=false,usedBufferSize=9984] 2023-07-15 11:55:29,218 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 11916 2023-07-15 11:55:29,222 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:55:29,253 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:55:29,257 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:29,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:29,367 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 17405 2023-07-15 11:55:29,367 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 51 WAL entries eligible for replication 2023-07-15 11:55:29,367 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 17405 2023-07-15 11:55:29,368 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/89b547022cb2e9610cb6246270f3cbaf/109=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/110=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/111=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/112=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/113=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/114=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/115=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/116=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/117=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/118=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/119=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/120=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/121=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/122=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/123=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/124=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/125=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/126=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/127=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/128=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/129=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/130=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/131=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/132=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/133=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/134=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/135=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/136=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/137=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/138=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/139=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/140=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/141=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/142=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/143=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/144=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/145=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/146=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/147=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/148=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/149=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/150=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/151=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/152=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/153=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/154=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/155=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/156=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/157=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/158=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/159=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228, lastWalPosition=17405, nbRowKeys=51, nbHFiles=0, heapSize=10608, lastSeqIds={}, endOfFile=false,usedBufferSize=10608] 2023-07-15 11:55:29,369 DEBUG [SinkThread-6] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:29,371 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54368, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-15 11:55:29,371 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:29,373 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39401] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:55:29,373 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39401] zookeeper.ReadOnlyZKClient(139): Connect 0x2a3ada9d to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:29,398 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39401] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@24fd31aa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:29,407 DEBUG [hconnection-0x20c2b1ed-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:29,412 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47940, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-15 11:55:29,419 DEBUG [hconnection-0x20c2b1ed-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:29,423 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39396, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-15 11:55:29,427 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39401] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:55:29,439 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:29,480 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:29,482 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:29,483 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 44 WAL entries eligible for replication 2023-07-15 11:55:29,483 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:29,483 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/89b547022cb2e9610cb6246270f3cbaf/160=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/161=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/162=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/163=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/164=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/165=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/166=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/167=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/168=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/169=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/170=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/171=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/172=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/173=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/174=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/175=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/176=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/177=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/178=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/179=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/180=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/181=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/182=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/183=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/184=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/185=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/186=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/187=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/188=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/189=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/190=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/191=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/192=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/193=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/194=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/195=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/196=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/197=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/198=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/199=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/200=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/201=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/202=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228, lastWalPosition=22269, nbRowKeys=44, nbHFiles=0, heapSize=9152, lastSeqIds={}, endOfFile=false,usedBufferSize=9152] 2023-07-15 11:55:29,485 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:55:29,490 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45535] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:55:29,512 INFO [jenkins-hbase4:46827Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:55:29,512 INFO [jenkins-hbase4:46827Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 97 2023-07-15 11:55:29,512 INFO [jenkins-hbase4:45535Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:55:29,512 INFO [jenkins-hbase4:45535Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 152 2023-07-15 11:55:29,517 INFO [jenkins-hbase4:43589Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:55:29,517 INFO [jenkins-hbase4:39401Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:55:29,517 INFO [jenkins-hbase4:39401Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 51 2023-07-15 11:55:29,548 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:29,588 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:29,593 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:29,656 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:29,700 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:29,707 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:29,768 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:29,808 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:29,815 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:29,876 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:29,915 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:29,922 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:29,959 DEBUG [hconnection-0x3525bc9c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:29,964 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47956, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:29,984 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:29,989 DEBUG [hconnection-0x3525bc9c-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:29,992 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39408, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:29,999 DEBUG [Listener at localhost/46025] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-07-15 11:55:29,999 INFO [Listener at localhost/46025] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-15 11:55:30,000 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1e6db056 to 127.0.0.1:57666 2023-07-15 11:55:30,000 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,001 DEBUG [Listener at localhost/46025] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-15 11:55:30,001 DEBUG [Listener at localhost/46025] util.JVMClusterUtil(257): Found active master hash=378665586, stopped=false 2023-07-15 11:55:30,001 INFO [Listener at localhost/46025] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:30,003 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-15 11:55:30,003 INFO [Listener at localhost/46025] procedure2.ProcedureExecutor(629): Stopping 2023-07-15 11:55:30,003 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-15 11:55:30,003 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-15 11:55:30,003 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-15 11:55:30,003 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7b6c49dd to 127.0.0.1:57666 2023-07-15 11:55:30,003 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-15 11:55:30,003 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:30,005 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,006 INFO [Listener at localhost/46025] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39401,1689422123877' ***** 2023-07-15 11:55:30,006 INFO [Listener at localhost/46025] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-15 11:55:30,006 INFO [Listener at localhost/46025] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46827,1689422123904' ***** 2023-07-15 11:55:30,006 INFO [Listener at localhost/46025] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-15 11:55:30,006 INFO [Listener at localhost/46025] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45535,1689422123929' ***** 2023-07-15 11:55:30,006 INFO [RS:1;jenkins-hbase4:46827] regionserver.HeapMemoryManager(220): Stopping 2023-07-15 11:55:30,006 INFO [RS:0;jenkins-hbase4:39401] regionserver.HeapMemoryManager(220): Stopping 2023-07-15 11:55:30,007 INFO [RS:1;jenkins-hbase4:46827] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-15 11:55:30,007 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-15 11:55:30,007 INFO [RS:1;jenkins-hbase4:46827] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-15 11:55:30,007 INFO [Listener at localhost/46025] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-15 11:55:30,007 INFO [RS:0;jenkins-hbase4:39401] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-15 11:55:30,009 INFO [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(3305): Received CLOSE for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:30,009 INFO [RS:2;jenkins-hbase4:45535] regionserver.HeapMemoryManager(220): Stopping 2023-07-15 11:55:30,009 INFO [RS:2;jenkins-hbase4:45535] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-15 11:55:30,009 INFO [RS:2;jenkins-hbase4:45535] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-15 11:55:30,009 INFO [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(3305): Received CLOSE for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:30,009 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-15 11:55:30,007 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-15 11:55:30,009 INFO [Listener at localhost/46025] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43589,1689422123957' ***** 2023-07-15 11:55:30,009 INFO [RS:0;jenkins-hbase4:39401] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-15 11:55:30,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8bb3eb49d5c769961a9b0f7da157f1d1, disabling compactions & flushes 2023-07-15 11:55:30,012 INFO [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(3305): Received CLOSE for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:30,011 INFO [Listener at localhost/46025] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-15 11:55:30,010 INFO [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:30,013 DEBUG [RS:1;jenkins-hbase4:46827] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3222ebea to 127.0.0.1:57666 2023-07-15 11:55:30,013 INFO [RS:3;jenkins-hbase4:43589] regionserver.HeapMemoryManager(220): Stopping 2023-07-15 11:55:30,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing faf8bf9c0e6d77a0bdf0f3a70ab43734, disabling compactions & flushes 2023-07-15 11:55:30,015 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:30,010 INFO [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:30,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:30,015 DEBUG [RS:2;jenkins-hbase4:45535] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57b7b3e7 to 127.0.0.1:57666 2023-07-15 11:55:30,015 INFO [RS:3;jenkins-hbase4:43589] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-15 11:55:30,015 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-15 11:55:30,013 DEBUG [RS:1;jenkins-hbase4:46827] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,013 INFO [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:30,017 INFO [RS:1;jenkins-hbase4:46827] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-15 11:55:30,017 DEBUG [RS:0;jenkins-hbase4:39401] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x15836c6a to 127.0.0.1:57666 2023-07-15 11:55:30,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ed49ae2049e592f2f5d8017bfcee4d2f, disabling compactions & flushes 2023-07-15 11:55:30,012 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:30,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:30,017 DEBUG [RS:0;jenkins-hbase4:39401] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,017 INFO [RS:1;jenkins-hbase4:46827] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-15 11:55:30,018 INFO [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-15 11:55:30,015 INFO [RS:3;jenkins-hbase4:43589] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-15 11:55:30,015 DEBUG [RS:2;jenkins-hbase4:45535] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:30,018 INFO [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-15 11:55:30,018 INFO [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:30,018 DEBUG [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1478): Online Regions={faf8bf9c0e6d77a0bdf0f3a70ab43734=hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.} 2023-07-15 11:55:30,018 INFO [RS:1;jenkins-hbase4:46827] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-15 11:55:30,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:30,018 INFO [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-15 11:55:30,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:30,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. after waiting 0 ms 2023-07-15 11:55:30,018 DEBUG [RS:3;jenkins-hbase4:43589] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7576e072 to 127.0.0.1:57666 2023-07-15 11:55:30,018 DEBUG [RS:3;jenkins-hbase4:43589] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,018 DEBUG [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1478): Online Regions={8bb3eb49d5c769961a9b0f7da157f1d1=t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.} 2023-07-15 11:55:30,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. after waiting 0 ms 2023-07-15 11:55:30,019 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-15 11:55:30,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:30,019 INFO [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43589,1689422123957; all regions closed. 2023-07-15 11:55:30,018 INFO [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-15 11:55:30,020 DEBUG [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1478): Online Regions={ed49ae2049e592f2f5d8017bfcee4d2f=t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f., 1588230740=hbase:meta,,1.1588230740} 2023-07-15 11:55:30,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:30,021 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing faf8bf9c0e6d77a0bdf0f3a70ab43734 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-15 11:55:30,021 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ed49ae2049e592f2f5d8017bfcee4d2f 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-15 11:55:30,021 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:30,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. after waiting 0 ms 2023-07-15 11:55:30,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:30,021 DEBUG [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1504): Waiting on 1588230740, ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:30,021 DEBUG [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1504): Waiting on faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:30,021 DEBUG [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1504): Waiting on 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:30,020 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-15 11:55:30,024 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-15 11:55:30,024 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-15 11:55:30,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8bb3eb49d5c769961a9b0f7da157f1d1 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-15 11:55:30,024 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-15 11:55:30,024 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-07-15 11:55:30,036 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:30,039 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:30,042 DEBUG [RS:3;jenkins-hbase4:43589] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:30,042 INFO [RS:3;jenkins-hbase4:43589] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C43589%2C1689422123957.jenkins-hbase4.apache.org%2C43589%2C1689422123957.regiongroup-0:(num 1689422124643) 2023-07-15 11:55:30,042 DEBUG [RS:3;jenkins-hbase4:43589] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,042 INFO [RS:3;jenkins-hbase4:43589] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:30,042 INFO [RS:3;jenkins-hbase4:43589] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-15 11:55:30,042 INFO [RS:3;jenkins-hbase4:43589] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-15 11:55:30,042 INFO [RS:3;jenkins-hbase4:43589] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-15 11:55:30,042 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:30,042 INFO [RS:3;jenkins-hbase4:43589] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-15 11:55:30,044 INFO [RS:3;jenkins-hbase4:43589] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43589 2023-07-15 11:55:30,086 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:30,088 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:30,088 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:30,105 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:30,111 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:30,111 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:30,111 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,111 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:30,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:30,111 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:30,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:30,112 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520018, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,111 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43589,1689422123957 2023-07-15 11:55:30,113 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,113 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-15 11:55:30,111 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:30,112 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:30,115 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43589,1689422123957] 2023-07-15 11:55:30,115 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43589,1689422123957; numProcessing=1 2023-07-15 11:55:30,117 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43589,1689422123957 already deleted, retry=false 2023-07-15 11:55:30,117 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43589,1689422123957 expired; onlineServers=3 2023-07-15 11:55:30,132 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/.tmp/cf1/d39c9fd504e448d9afb41abb769aeb63 2023-07-15 11:55:30,132 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/.tmp/info/b02e059c4d7e4d748077b55ccddf7346 2023-07-15 11:55:30,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/.tmp/cf1/b3a999ddc2d147879011cf288ac4fcc0 2023-07-15 11:55:30,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/.tmp/info/06116a296dcb48c687c514a4eb1cd99f 2023-07-15 11:55:30,142 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:30,145 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:30,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/.tmp/cf1/b3a999ddc2d147879011cf288ac4fcc0 as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1/b3a999ddc2d147879011cf288ac4fcc0 2023-07-15 11:55:30,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/.tmp/cf1/d39c9fd504e448d9afb41abb769aeb63 as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1/d39c9fd504e448d9afb41abb769aeb63 2023-07-15 11:55:30,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/.tmp/info/06116a296dcb48c687c514a4eb1cd99f as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/info/06116a296dcb48c687c514a4eb1cd99f 2023-07-15 11:55:30,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1/b3a999ddc2d147879011cf288ac4fcc0, entries=200, sequenceid=9, filesize=12.2 K 2023-07-15 11:55:30,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1/d39c9fd504e448d9afb41abb769aeb63, entries=100, sequenceid=7, filesize=8.3 K 2023-07-15 11:55:30,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/info/06116a296dcb48c687c514a4eb1cd99f, entries=2, sequenceid=6, filesize=4.8 K 2023-07-15 11:55:30,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for faf8bf9c0e6d77a0bdf0f3a70ab43734 in 170ms, sequenceid=6, compaction requested=false 2023-07-15 11:55:30,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for ed49ae2049e592f2f5d8017bfcee4d2f in 170ms, sequenceid=7, compaction requested=false 2023-07-15 11:55:30,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 8bb3eb49d5c769961a9b0f7da157f1d1 in 167ms, sequenceid=9, compaction requested=false 2023-07-15 11:55:30,195 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/.tmp/table/298f7c6b75f14089aed20c034ec773cc 2023-07-15 11:55:30,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-15 11:55:30,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-15 11:55:30,208 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/.tmp/info/b02e059c4d7e4d748077b55ccddf7346 as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info/b02e059c4d7e4d748077b55ccddf7346 2023-07-15 11:55:30,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-07-15 11:55:30,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:30,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:30,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for faf8bf9c0e6d77a0bdf0f3a70ab43734: 2023-07-15 11:55:30,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ed49ae2049e592f2f5d8017bfcee4d2f: 2023-07-15 11:55:30,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:30,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:30,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:30,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8bb3eb49d5c769961a9b0f7da157f1d1: 2023-07-15 11:55:30,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:30,211 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:30,217 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info/b02e059c4d7e4d748077b55ccddf7346, entries=30, sequenceid=19, filesize=8.1 K 2023-07-15 11:55:30,218 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/.tmp/table/298f7c6b75f14089aed20c034ec773cc as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table/298f7c6b75f14089aed20c034ec773cc 2023-07-15 11:55:30,224 DEBUG [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-15 11:55:30,224 INFO [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45535,1689422123929; all regions closed. 2023-07-15 11:55:30,224 INFO [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39401,1689422123877; all regions closed. 2023-07-15 11:55:30,224 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table/298f7c6b75f14089aed20c034ec773cc, entries=6, sequenceid=19, filesize=4.8 K 2023-07-15 11:55:30,224 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:30,228 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:30,232 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 207ms, sequenceid=19, compaction requested=false 2023-07-15 11:55:30,248 DEBUG [RS:2;jenkins-hbase4:45535] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:30,249 INFO [RS:2;jenkins-hbase4:45535] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45535%2C1689422123929.jenkins-hbase4.apache.org%2C45535%2C1689422123929.regiongroup-1:(num 1689422127796) 2023-07-15 11:55:30,251 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:30,255 DEBUG [RS:0;jenkins-hbase4:39401] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:30,255 INFO [RS:0;jenkins-hbase4:39401] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C39401%2C1689422123877.jenkins-hbase4.apache.org%2C39401%2C1689422123877.regiongroup-0:(num 1689422124634) 2023-07-15 11:55:30,261 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:30,261 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:30,261 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:30,269 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-15 11:55:30,270 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-15 11:55:30,270 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-15 11:55:30,271 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-15 11:55:30,271 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-15 11:55:30,272 DEBUG [RS:0;jenkins-hbase4:39401] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:30,272 INFO [RS:0;jenkins-hbase4:39401] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C39401%2C1689422123877.jenkins-hbase4.apache.org%2C39401%2C1689422123877.regiongroup-1:(num 1689422125238) 2023-07-15 11:55:30,272 DEBUG [RS:0;jenkins-hbase4:39401] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,272 INFO [RS:0;jenkins-hbase4:39401] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:30,272 INFO [RS:0;jenkins-hbase4:39401] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-15 11:55:30,273 INFO [RS:0;jenkins-hbase4:39401] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-15 11:55:30,273 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:30,273 INFO [RS:0;jenkins-hbase4:39401] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-15 11:55:30,273 INFO [RS:0;jenkins-hbase4:39401] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-15 11:55:30,275 DEBUG [RS:0;jenkins-hbase4:39401] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2a3ada9d to 127.0.0.1:57666 2023-07-15 11:55:30,275 DEBUG [RS:0;jenkins-hbase4:39401] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,276 INFO [RS:0;jenkins-hbase4:39401] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39401 2023-07-15 11:55:30,286 DEBUG [RS:2;jenkins-hbase4:45535] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:30,286 INFO [RS:2;jenkins-hbase4:45535] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45535%2C1689422123929.jenkins-hbase4.apache.org%2C45535%2C1689422123929.regiongroup-0:(num 1689422124625) 2023-07-15 11:55:30,286 DEBUG [RS:2;jenkins-hbase4:45535] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,286 INFO [RS:2;jenkins-hbase4:45535] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:30,286 INFO [RS:2;jenkins-hbase4:45535] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-15 11:55:30,286 INFO [RS:2;jenkins-hbase4:45535] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-15 11:55:30,286 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:30,286 INFO [RS:2;jenkins-hbase4:45535] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-15 11:55:30,286 INFO [RS:2;jenkins-hbase4:45535] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-15 11:55:30,287 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,287 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:30,287 DEBUG [RS:2;jenkins-hbase4:45535] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6c2b51a8 to 127.0.0.1:57666 2023-07-15 11:55:30,287 DEBUG [RS:2;jenkins-hbase4:45535] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,287 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:30,287 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520018, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,287 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,39401,1689422123877 2023-07-15 11:55:30,289 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39401,1689422123877] 2023-07-15 11:55:30,289 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-15 11:55:30,289 INFO [RS:2;jenkins-hbase4:45535] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45535 2023-07-15 11:55:30,289 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39401,1689422123877; numProcessing=2 2023-07-15 11:55:30,292 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,39401,1689422123877 already deleted, retry=false 2023-07-15 11:55:30,294 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39401,1689422123877 expired; onlineServers=2 2023-07-15 11:55:30,296 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:30,296 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,296 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520018, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,296 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45535,1689422123929 2023-07-15 11:55:30,296 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-15 11:55:30,313 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:30,313 INFO [RS:3;jenkins-hbase4:43589] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43589,1689422123957; zookeeper connection closed. 2023-07-15 11:55:30,313 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:43589-0x10168d63052000b, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:30,314 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3f783466] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3f783466 2023-07-15 11:55:30,320 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:30,368 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:30,374 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:30,394 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:30,394 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:39401-0x10168d630520008, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:30,394 INFO [RS:0;jenkins-hbase4:39401] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39401,1689422123877; zookeeper connection closed. 2023-07-15 11:55:30,396 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5c10f6ce] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5c10f6ce 2023-07-15 11:55:30,396 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45535,1689422123929] 2023-07-15 11:55:30,396 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45535,1689422123929; numProcessing=3 2023-07-15 11:55:30,398 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45535,1689422123929 already deleted, retry=false 2023-07-15 11:55:30,398 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45535,1689422123929 expired; onlineServers=1 2023-07-15 11:55:30,424 INFO [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46827,1689422123904; all regions closed. 2023-07-15 11:55:30,424 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:30,427 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:30,436 DEBUG [RS:1;jenkins-hbase4:46827] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:30,436 INFO [RS:1;jenkins-hbase4:46827] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C46827%2C1689422123904.meta:.meta(num 1689422124747) 2023-07-15 11:55:30,436 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:30,445 DEBUG [RS:1;jenkins-hbase4:46827] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:30,445 INFO [RS:1;jenkins-hbase4:46827] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C46827%2C1689422123904.jenkins-hbase4.apache.org%2C46827%2C1689422123904.regiongroup-0:(num 1689422124625) 2023-07-15 11:55:30,445 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:30,452 DEBUG [RS:1;jenkins-hbase4:46827] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:30,453 INFO [RS:1;jenkins-hbase4:46827] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C46827%2C1689422123904.jenkins-hbase4.apache.org%2C46827%2C1689422123904.regiongroup-1:(num 1689422127181) 2023-07-15 11:55:30,453 DEBUG [RS:1;jenkins-hbase4:46827] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,453 INFO [RS:1;jenkins-hbase4:46827] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:30,453 INFO [RS:1;jenkins-hbase4:46827] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-15 11:55:30,453 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:30,453 DEBUG [RS:1;jenkins-hbase4:46827] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x20246b24 to 127.0.0.1:57666 2023-07-15 11:55:30,453 DEBUG [RS:1;jenkins-hbase4:46827] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,455 INFO [RS:1;jenkins-hbase4:46827] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46827 2023-07-15 11:55:30,459 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46827,1689422123904 2023-07-15 11:55:30,459 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,459 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520018, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:30,459 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-15 11:55:30,460 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46827,1689422123904] 2023-07-15 11:55:30,460 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46827,1689422123904; numProcessing=4 2023-07-15 11:55:30,461 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46827,1689422123904 already deleted, retry=false 2023-07-15 11:55:30,461 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46827,1689422123904 expired; onlineServers=0 2023-07-15 11:55:30,461 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36081,1689422123834' ***** 2023-07-15 11:55:30,461 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-15 11:55:30,462 DEBUG [M:0;jenkins-hbase4:36081] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@12ae7c78, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:30,462 INFO [M:0;jenkins-hbase4:36081] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:30,462 INFO [M:0;jenkins-hbase4:36081] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36081,1689422123834; all regions closed. 2023-07-15 11:55:30,462 DEBUG [M:0;jenkins-hbase4:36081] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:30,462 DEBUG [M:0;jenkins-hbase4:36081] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-15 11:55:30,462 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-15 11:55:30,463 DEBUG [M:0;jenkins-hbase4:36081] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-15 11:55:30,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422124216] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422124216,5,FailOnTimeoutGroup] 2023-07-15 11:55:30,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422124216] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422124216,5,FailOnTimeoutGroup] 2023-07-15 11:55:30,464 INFO [M:0;jenkins-hbase4:36081] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-15 11:55:30,465 INFO [M:0;jenkins-hbase4:36081] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-15 11:55:30,465 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-15 11:55:30,465 INFO [M:0;jenkins-hbase4:36081] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-15 11:55:30,465 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:30,465 DEBUG [M:0;jenkins-hbase4:36081] master.HMaster(1512): Stopping service threads 2023-07-15 11:55:30,465 INFO [M:0;jenkins-hbase4:36081] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-15 11:55:30,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:30,466 ERROR [M:0;jenkins-hbase4:36081] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-15 11:55:30,467 INFO [M:0;jenkins-hbase4:36081] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-15 11:55:30,467 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-15 11:55:30,467 DEBUG [M:0;jenkins-hbase4:36081] zookeeper.ZKUtil(398): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-15 11:55:30,467 WARN [M:0;jenkins-hbase4:36081] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-15 11:55:30,467 INFO [M:0;jenkins-hbase4:36081] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-15 11:55:30,468 INFO [M:0;jenkins-hbase4:36081] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-15 11:55:30,469 DEBUG [M:0;jenkins-hbase4:36081] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-15 11:55:30,469 INFO [M:0;jenkins-hbase4:36081] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:30,469 DEBUG [M:0;jenkins-hbase4:36081] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:30,469 DEBUG [M:0;jenkins-hbase4:36081] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-15 11:55:30,469 DEBUG [M:0;jenkins-hbase4:36081] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:30,469 INFO [M:0;jenkins-hbase4:36081] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.40 KB heapSize=65.18 KB 2023-07-15 11:55:30,475 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:30,481 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:30,489 INFO [M:0;jenkins-hbase4:36081] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.40 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/87fe3d1a2e1b45d28817fd4987a3cba4 2023-07-15 11:55:30,496 DEBUG [M:0;jenkins-hbase4:36081] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/87fe3d1a2e1b45d28817fd4987a3cba4 as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/87fe3d1a2e1b45d28817fd4987a3cba4 2023-07-15 11:55:30,502 INFO [M:0;jenkins-hbase4:36081] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/87fe3d1a2e1b45d28817fd4987a3cba4, entries=14, sequenceid=116, filesize=8.1 K 2023-07-15 11:55:30,503 INFO [M:0;jenkins-hbase4:36081] regionserver.HRegion(2948): Finished flush of dataSize ~55.40 KB/56732, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 34ms, sequenceid=116, compaction requested=false 2023-07-15 11:55:30,506 INFO [M:0;jenkins-hbase4:36081] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:30,506 DEBUG [M:0;jenkins-hbase4:36081] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:30,506 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,36081,1689422123834 2023-07-15 11:55:30,511 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:30,511 INFO [M:0;jenkins-hbase4:36081] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-15 11:55:30,512 INFO [M:0;jenkins-hbase4:36081] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36081 2023-07-15 11:55:30,517 DEBUG [M:0;jenkins-hbase4:36081] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,36081,1689422123834 already deleted, retry=false 2023-07-15 11:55:30,533 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:30,581 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:30,586 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:30,639 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:30,689 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:30,692 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:30,749 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:30,796 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:30,798 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:30,815 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:30,815 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:36081-0x10168d630520007, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:30,815 INFO [M:0;jenkins-hbase4:36081] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36081,1689422123834; zookeeper connection closed. 2023-07-15 11:55:30,854 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:30,901 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:30,903 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:30,915 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:30,915 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x10168d630520009, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:30,915 INFO [RS:1;jenkins-hbase4:46827] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46827,1689422123904; zookeeper connection closed. 2023-07-15 11:55:30,917 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@28982939] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@28982939 2023-07-15 11:55:30,960 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:31,008 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:31,012 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:31,068 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:31,115 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-15 11:55:31,116 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:31,116 INFO [RS:2;jenkins-hbase4:45535] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45535,1689422123929; zookeeper connection closed. 2023-07-15 11:55:31,116 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:45535-0x10168d63052000a, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:31,117 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2f5679f5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2f5679f5 2023-07-15 11:55:31,118 INFO [Listener at localhost/46025] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-07-15 11:55:31,120 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-15 11:55:31,152 INFO [Listener at localhost/46025] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-15 11:55:31,152 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x328bb04a to 127.0.0.1:57666 2023-07-15 11:55:31,152 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:31,153 DEBUG [Listener at localhost/46025] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-15 11:55:31,153 DEBUG [Listener at localhost/46025] util.JVMClusterUtil(257): Found active master hash=403395478, stopped=false 2023-07-15 11:55:31,153 INFO [Listener at localhost/46025] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:31,155 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-15 11:55:31,155 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-15 11:55:31,155 INFO [Listener at localhost/46025] procedure2.ProcedureExecutor(629): Stopping 2023-07-15 11:55:31,156 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3ac0596f to 127.0.0.1:57666 2023-07-15 11:55:31,156 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:31,156 INFO [Listener at localhost/46025] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34419,1689422119824' ***** 2023-07-15 11:55:31,156 INFO [Listener at localhost/46025] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-15 11:55:31,158 INFO [Listener at localhost/46025] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,35805,1689422119867' ***** 2023-07-15 11:55:31,158 INFO [Listener at localhost/46025] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-15 11:55:31,155 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-15 11:55:31,158 INFO [RS:1;jenkins-hbase4:35805] regionserver.HeapMemoryManager(220): Stopping 2023-07-15 11:55:31,158 INFO [RS:1;jenkins-hbase4:35805] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-15 11:55:31,158 INFO [RS:1;jenkins-hbase4:35805] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-15 11:55:31,159 INFO [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:31,159 DEBUG [RS:1;jenkins-hbase4:35805] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5b780f41 to 127.0.0.1:57666 2023-07-15 11:55:31,159 DEBUG [RS:1;jenkins-hbase4:35805] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:31,159 INFO [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,35805,1689422119867; all regions closed. 2023-07-15 11:55:31,156 INFO [RS:0;jenkins-hbase4:34419] regionserver.HeapMemoryManager(220): Stopping 2023-07-15 11:55:31,156 INFO [jenkins-hbase4:35805Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0]: currently replicating from: hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867/jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0.1689422121276 at position: -1 2023-07-15 11:55:31,159 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-15 11:55:31,159 INFO [jenkins-hbase4:34419Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 300, current progress: walGroup [jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1]: currently replicating from: hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228 at position: 22269 walGroup [jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0]: currently replicating from: hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276 at position: 10975 2023-07-15 11:55:31,158 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-15 11:55:31,158 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:31,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-15 11:55:31,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-15 11:55:31,163 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-15 11:55:31,164 INFO [RS:0;jenkins-hbase4:34419] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-15 11:55:31,164 INFO [RS:0;jenkins-hbase4:34419] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-15 11:55:31,164 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:31,164 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(3305): Received CLOSE for 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:31,167 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(3305): Received CLOSE for f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:31,167 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(3305): Received CLOSE for eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:31,167 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:31,167 DEBUG [RS:0;jenkins-hbase4:34419] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x599aae77 to 127.0.0.1:57666 2023-07-15 11:55:31,167 DEBUG [RS:0;jenkins-hbase4:34419] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:31,167 INFO [RS:0;jenkins-hbase4:34419] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-15 11:55:31,167 INFO [RS:0;jenkins-hbase4:34419] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-15 11:55:31,167 INFO [RS:0;jenkins-hbase4:34419] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-15 11:55:31,171 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 89b547022cb2e9610cb6246270f3cbaf, disabling compactions & flushes 2023-07-15 11:55:31,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:31,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:31,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. after waiting 0 ms 2023-07-15 11:55:31,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:31,176 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-15 11:55:31,178 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-15 11:55:31,179 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1478): Online Regions={89b547022cb2e9610cb6246270f3cbaf=t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf., f1407b2bc354da8a7c08c95abb9ab378=hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378., 1588230740=hbase:meta,,1.1588230740, eee02c0767aa2be4d83f125ea78097e3=t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.} 2023-07-15 11:55:31,179 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:31,180 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1504): Waiting on 1588230740, 89b547022cb2e9610cb6246270f3cbaf, eee02c0767aa2be4d83f125ea78097e3, f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:31,179 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-15 11:55:31,186 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-15 11:55:31,186 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-15 11:55:31,186 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-15 11:55:31,186 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-15 11:55:31,186 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 89b547022cb2e9610cb6246270f3cbaf 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-07-15 11:55:31,186 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-07-15 11:55:31,194 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:31,199 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] wal.ProtobufLogReader(425): Encountered a malformed edit, seeking back to last good position in file, from 91 to 83 java.io.EOFException: Invalid PB, EOF? Ignoring; originalPosition=83, currentPosition=91 at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:354) at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:95) at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:83) at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.readNextEntryAndRecordReaderPosition(WALEntryStream.java:259) at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:173) at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:102) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.tryAdvanceStreamAndCreateWALBatch(ReplicationSourceWALReader.java:259) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:145) Caused by: org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: encoded_region_name, table_name, log_sequence_number, write_time at org.apache.hbase.thirdparty.com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(UninitializedMessageException.java:79) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.checkMessageInitialized(AbstractParser.java:68) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:232) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:237) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:48) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.parseDelimitedFrom(ProtobufUtil.java:3576) at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:348) ... 7 more 2023-07-15 11:55:31,201 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-15 11:55:31,202 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-15 11:55:31,231 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29160 2023-07-15 11:55:31,231 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-15 11:55:31,231 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29160 2023-07-15 11:55:31,309 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/89b547022cb2e9610cb6246270f3cbaf/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228, lastWalPosition=29160, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-07-15 11:55:31,321 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-15 11:55:31,321 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-15 11:55:31,321 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-15 11:55:31,321 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/eee02c0767aa2be4d83f125ea78097e3/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276, lastWalPosition=14321, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-07-15 11:55:31,351 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/.tmp/info/163103135f7e4a54a2604cb8c7667c56 2023-07-15 11:55:31,381 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1504): Waiting on 1588230740, 89b547022cb2e9610cb6246270f3cbaf, eee02c0767aa2be4d83f125ea78097e3, f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:31,399 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/.tmp/cf1/3d44f5295edd462dbbf3383b29306943 2023-07-15 11:55:31,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3d44f5295edd462dbbf3383b29306943 2023-07-15 11:55:31,428 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/.tmp/rep_barrier/aa1178b203d84561adbcd93c00f5a776 2023-07-15 11:55:31,431 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29160 2023-07-15 11:55:31,437 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-15 11:55:31,450 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/.tmp/norep/68d8ade59a834b269f027ba67c4f7b60 2023-07-15 11:55:31,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68d8ade59a834b269f027ba67c4f7b60 2023-07-15 11:55:31,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/.tmp/cf1/3d44f5295edd462dbbf3383b29306943 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/cf1/3d44f5295edd462dbbf3383b29306943 2023-07-15 11:55:31,471 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/.tmp/table/6e4f2bca2c9b4940b4ad448e0fb75048 2023-07-15 11:55:31,476 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3d44f5295edd462dbbf3383b29306943 2023-07-15 11:55:31,476 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/cf1/3d44f5295edd462dbbf3383b29306943, entries=200, sequenceid=206, filesize=11.9 K 2023-07-15 11:55:31,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/.tmp/norep/68d8ade59a834b269f027ba67c4f7b60 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/norep/68d8ade59a834b269f027ba67c4f7b60 2023-07-15 11:55:31,484 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/.tmp/info/163103135f7e4a54a2604cb8c7667c56 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/info/163103135f7e4a54a2604cb8c7667c56 2023-07-15 11:55:31,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68d8ade59a834b269f027ba67c4f7b60 2023-07-15 11:55:31,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/norep/68d8ade59a834b269f027ba67c4f7b60, entries=101, sequenceid=206, filesize=8.3 K 2023-07-15 11:55:31,493 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for 89b547022cb2e9610cb6246270f3cbaf in 318ms, sequenceid=206, compaction requested=false 2023-07-15 11:55:31,500 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/info/163103135f7e4a54a2604cb8c7667c56, entries=30, sequenceid=19, filesize=8.1 K 2023-07-15 11:55:31,505 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/.tmp/rep_barrier/aa1178b203d84561adbcd93c00f5a776 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/rep_barrier/aa1178b203d84561adbcd93c00f5a776 2023-07-15 11:55:31,527 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/rep_barrier/aa1178b203d84561adbcd93c00f5a776, entries=2, sequenceid=19, filesize=4.9 K 2023-07-15 11:55:31,532 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/.tmp/table/6e4f2bca2c9b4940b4ad448e0fb75048 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/table/6e4f2bca2c9b4940b4ad448e0fb75048 2023-07-15 11:55:31,546 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/table/6e4f2bca2c9b4940b4ad448e0fb75048, entries=6, sequenceid=19, filesize=4.8 K 2023-07-15 11:55:31,548 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 361ms, sequenceid=19, compaction requested=false 2023-07-15 11:55:31,550 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29775 2023-07-15 11:55:31,550 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-15 11:55:31,550 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29775 2023-07-15 11:55:31,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-07-15 11:55:31,555 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-15 11:55:31,568 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:31,568 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 89b547022cb2e9610cb6246270f3cbaf: 2023-07-15 11:55:31,568 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:31,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f1407b2bc354da8a7c08c95abb9ab378, disabling compactions & flushes 2023-07-15 11:55:31,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:31,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:31,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. after waiting 0 ms 2023-07-15 11:55:31,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:31,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing f1407b2bc354da8a7c08c95abb9ab378 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-15 11:55:31,588 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1504): Waiting on 1588230740, eee02c0767aa2be4d83f125ea78097e3, f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:31,595 DEBUG [RS:1;jenkins-hbase4:35805] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs 2023-07-15 11:55:31,595 INFO [RS:1;jenkins-hbase4:35805] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0:(num 1689422121276) 2023-07-15 11:55:31,595 DEBUG [RS:1;jenkins-hbase4:35805] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:31,596 INFO [RS:1;jenkins-hbase4:35805] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:31,597 INFO [RS:1;jenkins-hbase4:35805] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-15 11:55:31,598 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] regionserver.WALEntryStream(207): Couldn't get file length information about log hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867/jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0.1689422121276, it was closed cleanly currently replicating from: hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867/jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0.1689422121276 at position: 0 2023-07-15 11:55:31,599 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867/jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0.1689422121276 2023-07-15 11:55:31,599 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-15 11:55:31,601 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867/jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0.1689422121276, lastWalPosition=-1, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-07-15 11:55:31,601 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-15 11:55:31,602 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:31,602 INFO [RS:1;jenkins-hbase4:35805] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-15 11:55:31,604 INFO [RS:1;jenkins-hbase4:35805] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-15 11:55:31,604 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0.1689422121276] 2023-07-15 11:55:31,604 INFO [RS:1;jenkins-hbase4:35805] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-15 11:55:31,605 INFO [RS:1;jenkins-hbase4:35805] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-15 11:55:31,639 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-15 11:55:31,640 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-15 11:55:31,641 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-15 11:55:31,641 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-15 11:55:31,642 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-15 11:55:31,658 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 30020 2023-07-15 11:55:31,658 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-15 11:55:31,662 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-15 11:55:31,739 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520017, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:31,739 DEBUG [RS:1;jenkins-hbase4:35805] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3ddb30aa to 127.0.0.1:57666 2023-07-15 11:55:31,742 DEBUG [RS:1;jenkins-hbase4:35805] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:31,742 INFO [RS:1;jenkins-hbase4:35805] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C35805%2C1689422119867.jenkins-hbase4.apache.org%2C35805%2C1689422119867.regiongroup-0,1 terminated 2023-07-15 11:55:31,743 INFO [RS:1;jenkins-hbase4:35805] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35805 2023-07-15 11:55:31,747 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-15 11:55:31,747 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:31,747 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,35805,1689422119867 2023-07-15 11:55:31,748 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-15 11:55:31,748 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-15 11:55:31,749 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,35805,1689422119867] 2023-07-15 11:55:31,749 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35805,1689422119867; numProcessing=1 2023-07-15 11:55:31,751 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,35805,1689422119867 already deleted, retry=false 2023-07-15 11:55:31,751 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,35805,1689422119867 expired; onlineServers=1 2023-07-15 11:55:31,768 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-15 11:55:31,788 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1504): Waiting on eee02c0767aa2be4d83f125ea78097e3, f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:31,873 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-15 11:55:31,946 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:31,946 INFO [RS:1;jenkins-hbase4:35805] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,35805,1689422119867; zookeeper connection closed. 2023-07-15 11:55:31,946 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:35805-0x10168d630520002, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:31,947 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@55153a87] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@55153a87 2023-07-15 11:55:31,978 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-15 11:55:31,989 DEBUG [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1504): Waiting on eee02c0767aa2be4d83f125ea78097e3, f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:32,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/.tmp/info/7c1a732702e24f1f847a91dd73b1f9e1 2023-07-15 11:55:32,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/.tmp/info/7c1a732702e24f1f847a91dd73b1f9e1 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/info/7c1a732702e24f1f847a91dd73b1f9e1 2023-07-15 11:55:32,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/info/7c1a732702e24f1f847a91dd73b1f9e1, entries=2, sequenceid=6, filesize=4.8 K 2023-07-15 11:55:32,048 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for f1407b2bc354da8a7c08c95abb9ab378 in 476ms, sequenceid=6, compaction requested=false 2023-07-15 11:55:32,055 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-15 11:55:32,056 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:32,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f1407b2bc354da8a7c08c95abb9ab378: 2023-07-15 11:55:32,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:32,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing eee02c0767aa2be4d83f125ea78097e3, disabling compactions & flushes 2023-07-15 11:55:32,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:32,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:32,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. after waiting 0 ms 2023-07-15 11:55:32,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:32,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing eee02c0767aa2be4d83f125ea78097e3 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-07-15 11:55:32,065 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-15 11:55:32,080 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/.tmp/cf1/983cbb2db3c5484487fba3518cd32e6b 2023-07-15 11:55:32,085 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-15 11:55:32,085 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-15 11:55:32,085 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-15 11:55:32,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 983cbb2db3c5484487fba3518cd32e6b 2023-07-15 11:55:32,098 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/.tmp/norep/65c44ba5926d4820bfdf72628e0df5c8 2023-07-15 11:55:32,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65c44ba5926d4820bfdf72628e0df5c8 2023-07-15 11:55:32,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/.tmp/cf1/983cbb2db3c5484487fba3518cd32e6b as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/cf1/983cbb2db3c5484487fba3518cd32e6b 2023-07-15 11:55:32,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 983cbb2db3c5484487fba3518cd32e6b 2023-07-15 11:55:32,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/cf1/983cbb2db3c5484487fba3518cd32e6b, entries=100, sequenceid=106, filesize=8.2 K 2023-07-15 11:55:32,114 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/.tmp/norep/65c44ba5926d4820bfdf72628e0df5c8 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/norep/65c44ba5926d4820bfdf72628e0df5c8 2023-07-15 11:55:32,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65c44ba5926d4820bfdf72628e0df5c8 2023-07-15 11:55:32,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/norep/65c44ba5926d4820bfdf72628e0df5c8, entries=51, sequenceid=106, filesize=6.5 K 2023-07-15 11:55:32,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for eee02c0767aa2be4d83f125ea78097e3 in 64ms, sequenceid=106, compaction requested=false 2023-07-15 11:55:32,125 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-15 11:55:32,125 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-15 11:55:32,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-07-15 11:55:32,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:32,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for eee02c0767aa2be4d83f125ea78097e3: 2023-07-15 11:55:32,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:32,189 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34419,1689422119824; all regions closed. 2023-07-15 11:55:32,190 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:32,192 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-07-15 11:55:32,193 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-15 11:55:32,198 DEBUG [RS:0;jenkins-hbase4:34419] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs 2023-07-15 11:55:32,198 INFO [RS:0;jenkins-hbase4:34419] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C34419%2C1689422119824.meta:.meta(num 1689422121463) 2023-07-15 11:55:32,198 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:32,204 DEBUG [RS:0;jenkins-hbase4:34419] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs 2023-07-15 11:55:32,204 INFO [RS:0;jenkins-hbase4:34419] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0:(num 1689422121276) 2023-07-15 11:55:32,205 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:32,211 DEBUG [RS:0;jenkins-hbase4:34419] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs 2023-07-15 11:55:32,211 INFO [RS:0;jenkins-hbase4:34419] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1:(num 1689422122228) 2023-07-15 11:55:32,211 DEBUG [RS:0;jenkins-hbase4:34419] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:32,211 INFO [RS:0;jenkins-hbase4:34419] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:32,211 INFO [RS:0;jenkins-hbase4:34419] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-15 11:55:32,211 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:32,211 INFO [RS:0;jenkins-hbase4:34419] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-15 11:55:32,314 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520018, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:32,314 DEBUG [RS:0;jenkins-hbase4:34419] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3dbbe3eb to 127.0.0.1:57666 2023-07-15 11:55:32,314 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520018, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:32,314 DEBUG [RS:0;jenkins-hbase4:34419] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:32,515 INFO [RS:0;jenkins-hbase4:34419] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1 terminated 2023-07-15 11:55:32,516 INFO [RS:0;jenkins-hbase4:34419] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1 terminated 2023-07-15 11:55:32,517 INFO [RS:0;jenkins-hbase4:34419] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34419 2023-07-15 11:55:32,520 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:32,520 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-15 11:55:32,522 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34419,1689422119824] 2023-07-15 11:55:32,522 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34419,1689422119824; numProcessing=2 2023-07-15 11:55:32,523 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,34419,1689422119824 already deleted, retry=false 2023-07-15 11:55:32,523 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34419,1689422119824 expired; onlineServers=0 2023-07-15 11:55:32,523 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38223,1689422118510' ***** 2023-07-15 11:55:32,523 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-15 11:55:32,524 DEBUG [M:0;jenkins-hbase4:38223] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@593659ad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:32,524 INFO [M:0;jenkins-hbase4:38223] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:32,524 INFO [M:0;jenkins-hbase4:38223] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38223,1689422118510; all regions closed. 2023-07-15 11:55:32,524 DEBUG [M:0;jenkins-hbase4:38223] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:32,524 DEBUG [M:0;jenkins-hbase4:38223] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-15 11:55:32,525 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-15 11:55:32,525 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422120922] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422120922,5,FailOnTimeoutGroup] 2023-07-15 11:55:32,525 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422120913] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422120913,5,FailOnTimeoutGroup] 2023-07-15 11:55:32,525 DEBUG [M:0;jenkins-hbase4:38223] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-15 11:55:32,526 INFO [M:0;jenkins-hbase4:38223] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-15 11:55:32,526 INFO [M:0;jenkins-hbase4:38223] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-15 11:55:32,527 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-15 11:55:32,527 INFO [M:0;jenkins-hbase4:38223] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-15 11:55:32,527 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:32,527 DEBUG [M:0;jenkins-hbase4:38223] master.HMaster(1512): Stopping service threads 2023-07-15 11:55:32,527 INFO [M:0;jenkins-hbase4:38223] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-15 11:55:32,527 ERROR [M:0;jenkins-hbase4:38223] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-07-15 11:55:32,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-15 11:55:32,527 INFO [M:0;jenkins-hbase4:38223] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-15 11:55:32,528 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-15 11:55:32,528 DEBUG [M:0;jenkins-hbase4:38223] zookeeper.ZKUtil(398): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-15 11:55:32,528 WARN [M:0;jenkins-hbase4:38223] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-15 11:55:32,528 INFO [M:0;jenkins-hbase4:38223] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-15 11:55:32,529 INFO [M:0;jenkins-hbase4:38223] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-15 11:55:32,529 DEBUG [M:0;jenkins-hbase4:38223] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-15 11:55:32,529 INFO [M:0;jenkins-hbase4:38223] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:32,529 DEBUG [M:0;jenkins-hbase4:38223] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:32,529 DEBUG [M:0;jenkins-hbase4:38223] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-15 11:55:32,530 DEBUG [M:0;jenkins-hbase4:38223] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:32,530 INFO [M:0;jenkins-hbase4:38223] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.04 KB heapSize=72.60 KB 2023-07-15 11:55:32,540 INFO [M:0;jenkins-hbase4:38223] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.04 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ee49164dbdb445559382832f382eae1a 2023-07-15 11:55:32,546 DEBUG [M:0;jenkins-hbase4:38223] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ee49164dbdb445559382832f382eae1a as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ee49164dbdb445559382832f382eae1a 2023-07-15 11:55:32,551 INFO [M:0;jenkins-hbase4:38223] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ee49164dbdb445559382832f382eae1a, entries=17, sequenceid=136, filesize=8.5 K 2023-07-15 11:55:32,552 INFO [M:0;jenkins-hbase4:38223] regionserver.HRegion(2948): Finished flush of dataSize ~61.04 KB/62508, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 21ms, sequenceid=136, compaction requested=false 2023-07-15 11:55:32,553 INFO [M:0;jenkins-hbase4:38223] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:32,553 DEBUG [M:0;jenkins-hbase4:38223] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:32,553 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,38223,1689422118510 2023-07-15 11:55:32,557 INFO [M:0;jenkins-hbase4:38223] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-15 11:55:32,557 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:32,557 INFO [M:0;jenkins-hbase4:38223] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38223 2023-07-15 11:55:32,559 DEBUG [M:0;jenkins-hbase4:38223] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,38223,1689422118510 already deleted, retry=false 2023-07-15 11:55:32,622 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:32,622 INFO [RS:0;jenkins-hbase4:34419] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34419,1689422119824; zookeeper connection closed. 2023-07-15 11:55:32,622 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:34419-0x10168d630520001, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:32,623 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@48713a4a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@48713a4a 2023-07-15 11:55:32,623 INFO [Listener at localhost/46025] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-07-15 11:55:32,722 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:32,722 INFO [M:0;jenkins-hbase4:38223] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38223,1689422118510; zookeeper connection closed. 2023-07-15 11:55:32,723 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:38223-0x10168d630520000, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:32,725 INFO [Listener at localhost/46025] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-15 11:55:32,725 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:32,726 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:32,726 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:32,726 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:32,726 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:32,726 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:32,728 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35607 2023-07-15 11:55:32,728 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:32,729 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:32,730 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=master:35607 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:32,734 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:356070x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:32,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:35607-0x10168d63052001c connected 2023-07-15 11:55:32,737 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:32,737 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:32,738 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-15 11:55:32,738 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35607 2023-07-15 11:55:32,738 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35607 2023-07-15 11:55:32,739 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35607 2023-07-15 11:55:32,739 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35607 2023-07-15 11:55:32,739 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35607 2023-07-15 11:55:32,739 INFO [Listener at localhost/46025] master.HMaster(444): hbase.rootdir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346, hbase.cluster.distributed=false 2023-07-15 11:55:32,740 DEBUG [pool-456-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-15 11:55:32,752 INFO [Listener at localhost/46025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-15 11:55:32,752 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:32,752 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:32,752 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:32,752 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:32,752 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:32,752 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:32,755 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33403 2023-07-15 11:55:32,755 INFO [Listener at localhost/46025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-15 11:55:32,756 DEBUG [Listener at localhost/46025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-15 11:55:32,756 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:32,757 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:32,758 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33403 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:32,762 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:334030x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:32,763 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:334030x0, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:32,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33403-0x10168d63052001d connected 2023-07-15 11:55:32,764 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:32,765 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-15 11:55:32,766 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33403 2023-07-15 11:55:32,766 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33403 2023-07-15 11:55:32,766 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33403 2023-07-15 11:55:32,769 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33403 2023-07-15 11:55:32,769 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33403 2023-07-15 11:55:32,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,774 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-15 11:55:32,775 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,776 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-15 11:55:32,776 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-15 11:55:32,776 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:32,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-15 11:55:32,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-15 11:55:32,778 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,35607,1689422132725 from backup master directory 2023-07-15 11:55:32,779 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,779 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-15 11:55:32,779 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:32,779 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:32,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x100c031b to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:32,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@57604ae1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:32,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:32,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-15 11:55:32,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:32,837 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:32,844 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,36081,1689422123834 to hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,36081,1689422123834-dead as it is dead 2023-07-15 11:55:32,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,36081,1689422123834-dead/jenkins-hbase4.apache.org%2C36081%2C1689422123834.jenkins-hbase4.apache.org%2C36081%2C1689422123834.regiongroup-0.1689422124088 2023-07-15 11:55:32,849 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,36081,1689422123834-dead/jenkins-hbase4.apache.org%2C36081%2C1689422123834.jenkins-hbase4.apache.org%2C36081%2C1689422123834.regiongroup-0.1689422124088 after 4ms 2023-07-15 11:55:32,850 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,36081,1689422123834-dead/jenkins-hbase4.apache.org%2C36081%2C1689422123834.jenkins-hbase4.apache.org%2C36081%2C1689422123834.regiongroup-0.1689422124088 to hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36081%2C1689422123834.jenkins-hbase4.apache.org%2C36081%2C1689422123834.regiongroup-0.1689422124088 2023-07-15 11:55:32,850 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,36081,1689422123834-dead 2023-07-15 11:55:32,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:32,853 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35607%2C1689422132725.jenkins-hbase4.apache.org%2C35607%2C1689422132725.regiongroup-0, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,35607,1689422132725, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/oldWALs, maxLogs=10 2023-07-15 11:55:32,865 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,35607,1689422132725/jenkins-hbase4.apache.org%2C35607%2C1689422132725.jenkins-hbase4.apache.org%2C35607%2C1689422132725.regiongroup-0.1689422132853 2023-07-15 11:55:32,865 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33417,DS-274df9f8-c308-48fc-8592-c230a3934cdd,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK]] 2023-07-15 11:55:32,866 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:32,866 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:32,866 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:32,866 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:32,870 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:32,871 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-15 11:55:32,871 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-15 11:55:32,878 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/87fe3d1a2e1b45d28817fd4987a3cba4 2023-07-15 11:55:32,879 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:32,880 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-15 11:55:32,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36081%2C1689422123834.jenkins-hbase4.apache.org%2C36081%2C1689422123834.regiongroup-0.1689422124088 2023-07-15 11:55:32,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36081%2C1689422123834.jenkins-hbase4.apache.org%2C36081%2C1689422123834.regiongroup-0.1689422124088 2023-07-15 11:55:32,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36081%2C1689422123834.jenkins-hbase4.apache.org%2C36081%2C1689422123834.regiongroup-0.1689422124088 2023-07-15 11:55:32,891 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:32,893 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-07-15 11:55:32,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69765354, jitterRate=0.03958478569984436}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-15 11:55:32,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:32,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-15 11:55:32,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-15 11:55:32,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-15 11:55:32,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-15 11:55:32,897 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-15 11:55:32,909 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-15 11:55:32,910 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-15 11:55:32,910 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-15 11:55:32,910 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-15 11:55:32,910 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-15 11:55:32,911 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-15 11:55:32,911 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 14 msec 2023-07-15 11:55:32,911 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-15 11:55:32,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-15 11:55:32,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46827,1689422123904, table=hbase:meta, region=1588230740 2023-07-15 11:55:32,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-15 11:55:32,916 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46827,1689422123904 already deleted, retry=false 2023-07-15 11:55:32,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46827,1689422123904 on jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,929 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46827,1689422123904, splitWal=true, meta=true 2023-07-15 11:55:32,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,46827,1689422123904 (carryingMeta=true) jenkins-hbase4.apache.org,46827,1689422123904/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@6fe2a42b[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-15 11:55:32,931 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,39401,1689422123877 already deleted, retry=false 2023-07-15 11:55:32,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,39401,1689422123877 on jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,39401,1689422123877, splitWal=true, meta=false 2023-07-15 11:55:32,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase4.apache.org,39401,1689422123877 (carryingMeta=false) jenkins-hbase4.apache.org,39401,1689422123877/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@6539adf1[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-15 11:55:32,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43589,1689422123957 already deleted, retry=false 2023-07-15 11:55:32,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,43589,1689422123957 on jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,934 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,43589,1689422123957, splitWal=true, meta=false 2023-07-15 11:55:32,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase4.apache.org,43589,1689422123957 (carryingMeta=false) jenkins-hbase4.apache.org,43589,1689422123957/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@1f008e81[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-15 11:55:32,937 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45535,1689422123929 already deleted, retry=false 2023-07-15 11:55:32,937 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45535,1689422123929 on jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,938 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45535,1689422123929, splitWal=true, meta=false 2023-07-15 11:55:32,938 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,45535,1689422123929 (carryingMeta=false) jenkins-hbase4.apache.org,45535,1689422123929/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@1a4bf341[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-15 11:55:32,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-15 11:55:32,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-15 11:55:32,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-15 11:55:32,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-15 11:55:32,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-15 11:55:32,950 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-15 11:55:32,950 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-15 11:55:32,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-15 11:55:32,952 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-15 11:55:32,952 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-15 11:55:32,952 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:32,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,35607,1689422132725, sessionid=0x10168d63052001c, setting cluster-up flag (Was=false) 2023-07-15 11:55:32,957 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-15 11:55:32,958 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-15 11:55:32,961 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:32,961 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.hbase-snapshot/.tmp 2023-07-15 11:55:32,962 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:32,962 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:32,962 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:32,962 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:32,962 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-15 11:55:32,962 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:32,962 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:32,962 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:32,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689422162963 2023-07-15 11:55:32,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-15 11:55:32,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-15 11:55:32,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-15 11:55:32,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-15 11:55:32,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-15 11:55:32,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-15 11:55:32,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:32,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-15 11:55:32,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-15 11:55:32,968 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-15 11:55:32,968 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-15 11:55:32,968 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-15 11:55:32,968 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422132968,5,FailOnTimeoutGroup] 2023-07-15 11:55:32,968 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422132968,5,FailOnTimeoutGroup] 2023-07-15 11:55:32,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:32,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-15 11:55:32,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:32,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:32,969 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46827,1689422123904; numProcessing=1 2023-07-15 11:55:32,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689422132969, completionTime=-1 2023-07-15 11:55:32,969 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43589,1689422123957; numProcessing=2 2023-07-15 11:55:32,969 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-15 11:55:32,969 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45535,1689422123929; numProcessing=3 2023-07-15 11:55:32,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-15 11:55:32,970 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39401,1689422123877; numProcessing=4 2023-07-15 11:55:32,970 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43589,1689422123957, splitWal=true, meta=false 2023-07-15 11:55:32,970 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46827,1689422123904, splitWal=true, meta=true 2023-07-15 11:55:32,970 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,39401,1689422123877, splitWal=true, meta=false 2023-07-15 11:55:32,970 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45535,1689422123929, splitWal=true, meta=false 2023-07-15 11:55:32,971 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(951): ClusterId : e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:32,972 DEBUG [RS:0;jenkins-hbase4:33403] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-15 11:55:32,972 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46827,1689422123904, splitWal=true, meta=true, isMeta: true 2023-07-15 11:55:32,974 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904-splitting 2023-07-15 11:55:32,975 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904-splitting dir is empty, no logs to split. 2023-07-15 11:55:32,975 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46827,1689422123904 WAL count=0, meta=true 2023-07-15 11:55:32,979 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904-splitting dir is empty, no logs to split. 2023-07-15 11:55:32,979 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46827,1689422123904 WAL count=0, meta=true 2023-07-15 11:55:32,979 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46827,1689422123904 WAL splitting is done? wals=0, meta=true 2023-07-15 11:55:32,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-15 11:55:32,987 DEBUG [RS:0;jenkins-hbase4:33403] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-15 11:55:32,987 DEBUG [RS:0;jenkins-hbase4:33403] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-15 11:55:32,987 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-15 11:55:32,989 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:55:32,990 DEBUG [RS:0;jenkins-hbase4:33403] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-15 11:55:32,992 DEBUG [RS:0;jenkins-hbase4:33403] zookeeper.ReadOnlyZKClient(139): Connect 0x5651b708 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:32,998 DEBUG [RS:0;jenkins-hbase4:33403] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@589653b2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:32,998 DEBUG [RS:0;jenkins-hbase4:33403] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ca9dc78, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:33,006 DEBUG [RS:0;jenkins-hbase4:33403] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:33403 2023-07-15 11:55:33,006 INFO [RS:0;jenkins-hbase4:33403] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-15 11:55:33,006 INFO [RS:0;jenkins-hbase4:33403] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-15 11:55:33,006 DEBUG [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1022): About to register with Master. 2023-07-15 11:55:33,007 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,35607,1689422132725 with isa=jenkins-hbase4.apache.org/172.31.14.131:33403, startcode=1689422132751 2023-07-15 11:55:33,007 DEBUG [RS:0;jenkins-hbase4:33403] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-15 11:55:33,010 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34867, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-15 11:55:33,011 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=35607] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:33,012 DEBUG [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346 2023-07-15 11:55:33,012 DEBUG [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46809 2023-07-15 11:55:33,012 DEBUG [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-15 11:55:33,017 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:33,017 DEBUG [RS:0;jenkins-hbase4:33403] zookeeper.ZKUtil(162): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:33,018 WARN [RS:0;jenkins-hbase4:33403] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:33,018 INFO [RS:0;jenkins-hbase4:33403] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:33,018 INFO [RS:0;jenkins-hbase4:33403] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:33,018 DEBUG [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:33,018 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33403,1689422132751] 2023-07-15 11:55:33,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-15 11:55:33,021 DEBUG [RS:0;jenkins-hbase4:33403] zookeeper.ZKUtil(162): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:33,022 DEBUG [RS:0;jenkins-hbase4:33403] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:55:33,022 INFO [RS:0;jenkins-hbase4:33403] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-15 11:55:33,024 INFO [RS:0;jenkins-hbase4:33403] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-15 11:55:33,024 INFO [RS:0;jenkins-hbase4:33403] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-15 11:55:33,024 INFO [RS:0;jenkins-hbase4:33403] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:33,024 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-15 11:55:33,026 INFO [RS:0;jenkins-hbase4:33403] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:33,026 DEBUG [RS:0;jenkins-hbase4:33403] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:33,026 DEBUG [RS:0;jenkins-hbase4:33403] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:33,026 DEBUG [RS:0;jenkins-hbase4:33403] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:33,026 DEBUG [RS:0;jenkins-hbase4:33403] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:33,026 DEBUG [RS:0;jenkins-hbase4:33403] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:33,026 DEBUG [RS:0;jenkins-hbase4:33403] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:33,026 DEBUG [RS:0;jenkins-hbase4:33403] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:33,026 DEBUG [RS:0;jenkins-hbase4:33403] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:33,026 DEBUG [RS:0;jenkins-hbase4:33403] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:33,026 DEBUG [RS:0;jenkins-hbase4:33403] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:33,028 INFO [RS:0;jenkins-hbase4:33403] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:33,028 INFO [RS:0;jenkins-hbase4:33403] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:33,028 INFO [RS:0;jenkins-hbase4:33403] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:33,040 INFO [RS:0;jenkins-hbase4:33403] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-15 11:55:33,040 INFO [RS:0;jenkins-hbase4:33403] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33403,1689422132751-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:33,050 INFO [RS:0;jenkins-hbase4:33403] regionserver.Replication(203): jenkins-hbase4.apache.org,33403,1689422132751 started 2023-07-15 11:55:33,050 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,33403,1689422132751, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33403, sessionid=0x10168d63052001d 2023-07-15 11:55:33,050 DEBUG [RS:0;jenkins-hbase4:33403] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-15 11:55:33,051 DEBUG [RS:0;jenkins-hbase4:33403] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:33,051 DEBUG [RS:0;jenkins-hbase4:33403] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33403,1689422132751' 2023-07-15 11:55:33,051 DEBUG [RS:0;jenkins-hbase4:33403] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-15 11:55:33,051 DEBUG [RS:0;jenkins-hbase4:33403] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-15 11:55:33,051 DEBUG [RS:0;jenkins-hbase4:33403] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-15 11:55:33,051 DEBUG [RS:0;jenkins-hbase4:33403] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-15 11:55:33,051 DEBUG [RS:0;jenkins-hbase4:33403] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:33,051 DEBUG [RS:0;jenkins-hbase4:33403] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33403,1689422132751' 2023-07-15 11:55:33,051 DEBUG [RS:0;jenkins-hbase4:33403] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-15 11:55:33,052 DEBUG [RS:0;jenkins-hbase4:33403] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-15 11:55:33,052 DEBUG [RS:0;jenkins-hbase4:33403] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-15 11:55:33,052 INFO [RS:0;jenkins-hbase4:33403] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-15 11:55:33,052 INFO [RS:0;jenkins-hbase4:33403] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-15 11:55:33,139 DEBUG [jenkins-hbase4:35607] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-15 11:55:33,141 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33403,1689422132751, state=OPENING 2023-07-15 11:55:33,142 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-15 11:55:33,143 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-15 11:55:33,143 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33403,1689422132751}] 2023-07-15 11:55:33,152 INFO [RS:0;jenkins-hbase4:33403] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:33,154 INFO [RS:0;jenkins-hbase4:33403] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33403%2C1689422132751.jenkins-hbase4.apache.org%2C33403%2C1689422132751.regiongroup-0, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:33,166 INFO [RS:0;jenkins-hbase4:33403] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751/jenkins-hbase4.apache.org%2C33403%2C1689422132751.jenkins-hbase4.apache.org%2C33403%2C1689422132751.regiongroup-0.1689422133155 2023-07-15 11:55:33,166 DEBUG [RS:0;jenkins-hbase4:33403] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33417,DS-274df9f8-c308-48fc-8592-c230a3934cdd,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK]] 2023-07-15 11:55:33,298 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:33,298 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:33,301 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52032, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-15 11:55:33,305 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-15 11:55:33,305 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:33,305 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:33,305 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:33,307 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33403%2C1689422132751.meta, suffix=.meta, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:33,317 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751/jenkins-hbase4.apache.org%2C33403%2C1689422132751.meta.1689422133307.meta 2023-07-15 11:55:33,317 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45131,DS-caa7d3ae-4d76-4779-b495-05a972fbb935,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-274df9f8-c308-48fc-8592-c230a3934cdd,DISK]] 2023-07-15 11:55:33,317 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:33,317 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-15 11:55:33,317 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-15 11:55:33,318 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-15 11:55:33,318 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-15 11:55:33,318 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:33,318 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-15 11:55:33,318 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-15 11:55:33,320 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-15 11:55:33,320 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info 2023-07-15 11:55:33,321 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info 2023-07-15 11:55:33,321 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-15 11:55:33,327 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info/b02e059c4d7e4d748077b55ccddf7346 2023-07-15 11:55:33,327 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:33,327 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-15 11:55:33,328 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:33,328 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:33,329 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-15 11:55:33,329 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:33,329 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-15 11:55:33,330 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table 2023-07-15 11:55:33,330 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table 2023-07-15 11:55:33,330 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-15 11:55:33,336 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table/298f7c6b75f14089aed20c034ec773cc 2023-07-15 11:55:33,337 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:33,338 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740 2023-07-15 11:55:33,339 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740 2023-07-15 11:55:33,342 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-15 11:55:33,344 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-15 11:55:33,344 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66066807, jitterRate=-0.015527859330177307}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-15 11:55:33,345 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-15 11:55:33,349 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1689422133298 2023-07-15 11:55:33,354 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-15 11:55:33,354 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-15 11:55:33,355 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33403,1689422132751, state=OPEN 2023-07-15 11:55:33,356 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-15 11:55:33,356 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-15 11:55:33,359 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-15 11:55:33,359 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33403,1689422132751 in 213 msec 2023-07-15 11:55:33,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=15 2023-07-15 11:55:33,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 378 msec 2023-07-15 11:55:34,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1553ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-15 11:55:36,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-15 11:55:37,056 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-15 11:55:37,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-15 11:55:37,478 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-15 11:55:37,481 DEBUG [hconnection-0x31b7a90f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:37,485 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52034, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:37,493 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=faf8bf9c0e6d77a0bdf0f3a70ab43734, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39401,1689422123877, regionLocation=jenkins-hbase4.apache.org,39401,1689422123877, openSeqNum=2 2023-07-15 11:55:37,494 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ed49ae2049e592f2f5d8017bfcee4d2f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46827,1689422123904, regionLocation=jenkins-hbase4.apache.org,46827,1689422123904, openSeqNum=2 2023-07-15 11:55:37,494 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=8bb3eb49d5c769961a9b0f7da157f1d1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45535,1689422123929, regionLocation=jenkins-hbase4.apache.org,45535,1689422123929, openSeqNum=2 2023-07-15 11:55:37,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-15 11:55:37,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689422197494 2023-07-15 11:55:37,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689422257494 2023-07-15 11:55:37,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 16 msec 2023-07-15 11:55:37,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35607,1689422132725-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:37,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35607,1689422132725-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:37,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35607,1689422132725-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:37,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:35607, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:37,516 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,43589,1689422123957 had 0 regions 2023-07-15 11:55:37,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:37,516 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46827,1689422123904 had 2 regions 2023-07-15 11:55:37,516 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,39401,1689422123877 had 1 regions 2023-07-15 11:55:37,516 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45535,1689422123929 had 1 regions 2023-07-15 11:55:37,517 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. is NOT online; state={faf8bf9c0e6d77a0bdf0f3a70ab43734 state=OPEN, ts=1689422137493, server=jenkins-hbase4.apache.org,39401,1689422123877}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-15 11:55:37,518 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43589,1689422123957, splitWal=true, meta=false, isMeta: false 2023-07-15 11:55:37,518 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,39401,1689422123877, splitWal=true, meta=false, isMeta: false 2023-07-15 11:55:37,519 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46827,1689422123904, splitWal=true, meta=true, isMeta: false 2023-07-15 11:55:37,518 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45535,1689422123929, splitWal=true, meta=false, isMeta: false 2023-07-15 11:55:37,521 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,43589,1689422123957-splitting 2023-07-15 11:55:37,522 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877-splitting 2023-07-15 11:55:37,522 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,43589,1689422123957-splitting dir is empty, no logs to split. 2023-07-15 11:55:37,523 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,43589,1689422123957 WAL count=0, meta=false 2023-07-15 11:55:37,523 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,39401,1689422123877/hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734., unknown_server=jenkins-hbase4.apache.org,46827,1689422123904/t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f., unknown_server=jenkins-hbase4.apache.org,45535,1689422123929/t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:37,524 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877-splitting dir is empty, no logs to split. 2023-07-15 11:55:37,524 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,39401,1689422123877 WAL count=0, meta=false 2023-07-15 11:55:37,524 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904-splitting dir is empty, no logs to split. 2023-07-15 11:55:37,525 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,46827,1689422123904 WAL count=0, meta=false 2023-07-15 11:55:37,525 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929-splitting 2023-07-15 11:55:37,526 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929-splitting dir is empty, no logs to split. 2023-07-15 11:55:37,526 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,45535,1689422123929 WAL count=0, meta=false 2023-07-15 11:55:37,527 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,43589,1689422123957-splitting dir is empty, no logs to split. 2023-07-15 11:55:37,527 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,43589,1689422123957 WAL count=0, meta=false 2023-07-15 11:55:37,527 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43589,1689422123957 WAL splitting is done? wals=0, meta=false 2023-07-15 11:55:37,529 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877-splitting dir is empty, no logs to split. 2023-07-15 11:55:37,529 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,39401,1689422123877 WAL count=0, meta=false 2023-07-15 11:55:37,529 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,39401,1689422123877 WAL splitting is done? wals=0, meta=false 2023-07-15 11:55:37,530 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,46827,1689422123904-splitting dir is empty, no logs to split. 2023-07-15 11:55:37,530 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,46827,1689422123904 WAL count=0, meta=false 2023-07-15 11:55:37,530 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46827,1689422123904 WAL splitting is done? wals=0, meta=false 2023-07-15 11:55:37,531 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929-splitting dir is empty, no logs to split. 2023-07-15 11:55:37,531 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,45535,1689422123929 WAL count=0, meta=false 2023-07-15 11:55:37,531 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,43589,1689422123957 failed, ignore...File hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,43589,1689422123957-splitting does not exist. 2023-07-15 11:55:37,531 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45535,1689422123929 WAL splitting is done? wals=0, meta=false 2023-07-15 11:55:37,531 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,39401,1689422123877 failed, ignore...File hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,39401,1689422123877-splitting does not exist. 2023-07-15 11:55:37,533 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN}] 2023-07-15 11:55:37,534 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN}] 2023-07-15 11:55:37,534 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,43589,1689422123957 after splitting done 2023-07-15 11:55:37,534 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,43589,1689422123957 from processing; numProcessing=3 2023-07-15 11:55:37,535 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN 2023-07-15 11:55:37,535 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,45535,1689422123929 failed, ignore...File hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,45535,1689422123929-splitting does not exist. 2023-07-15 11:55:37,535 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN 2023-07-15 11:55:37,536 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:55:37,536 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN}] 2023-07-15 11:55:37,536 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:55:37,537 DEBUG [jenkins-hbase4:35607] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-15 11:55:37,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43589,1689422123957, splitWal=true, meta=false in 4.6020 sec 2023-07-15 11:55:37,539 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=ed49ae2049e592f2f5d8017bfcee4d2f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:37,539 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=faf8bf9c0e6d77a0bdf0f3a70ab43734, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:37,539 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN 2023-07-15 11:55:37,540 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422137539"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422137539"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422137539"}]},"ts":"1689422137539"} 2023-07-15 11:55:37,540 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422137539"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422137539"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422137539"}]},"ts":"1689422137539"} 2023-07-15 11:55:37,541 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:55:37,543 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=21, state=RUNNABLE; OpenRegionProcedure ed49ae2049e592f2f5d8017bfcee4d2f, server=jenkins-hbase4.apache.org,33403,1689422132751}] 2023-07-15 11:55:37,544 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure faf8bf9c0e6d77a0bdf0f3a70ab43734, server=jenkins-hbase4.apache.org,33403,1689422132751}] 2023-07-15 11:55:37,693 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=8bb3eb49d5c769961a9b0f7da157f1d1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:37,693 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422137693"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422137693"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422137693"}]},"ts":"1689422137693"} 2023-07-15 11:55:37,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure 8bb3eb49d5c769961a9b0f7da157f1d1, server=jenkins-hbase4.apache.org,33403,1689422132751}] 2023-07-15 11:55:37,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:37,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:37,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33403%2C1689422132751.jenkins-hbase4.apache.org%2C33403%2C1689422132751.regiongroup-1, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:37,701 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:37,701 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => faf8bf9c0e6d77a0bdf0f3a70ab43734, NAME => 'hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:37,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:37,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:37,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:37,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:37,703 INFO [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:37,704 DEBUG [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/info 2023-07-15 11:55:37,704 DEBUG [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/info 2023-07-15 11:55:37,705 INFO [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region faf8bf9c0e6d77a0bdf0f3a70ab43734 columnFamilyName info 2023-07-15 11:55:37,711 DEBUG [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/info/06116a296dcb48c687c514a4eb1cd99f 2023-07-15 11:55:37,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751/jenkins-hbase4.apache.org%2C33403%2C1689422132751.jenkins-hbase4.apache.org%2C33403%2C1689422132751.regiongroup-1.1689422137701 2023-07-15 11:55:37,713 INFO [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] regionserver.HStore(310): Store=faf8bf9c0e6d77a0bdf0f3a70ab43734/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:37,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33417,DS-274df9f8-c308-48fc-8592-c230a3934cdd,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK]] 2023-07-15 11:55:37,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ed49ae2049e592f2f5d8017bfcee4d2f, NAME => 't1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:37,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:37,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:37,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:37,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:37,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:37,715 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:37,715 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:37,716 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1 2023-07-15 11:55:37,716 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1 2023-07-15 11:55:37,717 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed49ae2049e592f2f5d8017bfcee4d2f columnFamilyName cf1 2023-07-15 11:55:37,719 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:37,721 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened faf8bf9c0e6d77a0bdf0f3a70ab43734; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60084775, jitterRate=-0.10466708242893219}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-15 11:55:37,721 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for faf8bf9c0e6d77a0bdf0f3a70ab43734: 2023-07-15 11:55:37,726 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734., pid=25, masterSystemTime=1689422137695 2023-07-15 11:55:37,729 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1/d39c9fd504e448d9afb41abb769aeb63 2023-07-15 11:55:37,729 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(310): Store=ed49ae2049e592f2f5d8017bfcee4d2f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:37,729 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:37,729 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:37,729 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:37,730 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/norep 2023-07-15 11:55:37,730 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/norep 2023-07-15 11:55:37,731 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=faf8bf9c0e6d77a0bdf0f3a70ab43734, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:37,731 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422137730"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422137730"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422137730"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422137730"}]},"ts":"1689422137730"} 2023-07-15 11:55:37,731 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed49ae2049e592f2f5d8017bfcee4d2f columnFamilyName norep 2023-07-15 11:55:37,732 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(310): Store=ed49ae2049e592f2f5d8017bfcee4d2f/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:37,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:37,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:37,736 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-15 11:55:37,736 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure faf8bf9c0e6d77a0bdf0f3a70ab43734, server=jenkins-hbase4.apache.org,33403,1689422132751 in 189 msec 2023-07-15 11:55:37,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-15 11:55:37,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:37,739 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=16 2023-07-15 11:55:37,739 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,39401,1689422123877 after splitting done 2023-07-15 11:55:37,739 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN in 202 msec 2023-07-15 11:55:37,739 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,39401,1689422123877 from processing; numProcessing=2 2023-07-15 11:55:37,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ed49ae2049e592f2f5d8017bfcee4d2f; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61288868, jitterRate=-0.08672469854354858}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-15 11:55:37,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ed49ae2049e592f2f5d8017bfcee4d2f: 2023-07-15 11:55:37,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f., pid=24, masterSystemTime=1689422137695 2023-07-15 11:55:37,743 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,39401,1689422123877, splitWal=true, meta=false in 4.8080 sec 2023-07-15 11:55:37,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:37,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:37,744 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=ed49ae2049e592f2f5d8017bfcee4d2f, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:37,744 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422137744"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422137744"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422137744"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422137744"}]},"ts":"1689422137744"} 2023-07-15 11:55:37,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=21 2023-07-15 11:55:37,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=21, state=SUCCESS; OpenRegionProcedure ed49ae2049e592f2f5d8017bfcee4d2f, server=jenkins-hbase4.apache.org,33403,1689422132751 in 203 msec 2023-07-15 11:55:37,751 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=15 2023-07-15 11:55:37,751 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46827,1689422123904 after splitting done 2023-07-15 11:55:37,751 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN in 215 msec 2023-07-15 11:55:37,751 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46827,1689422123904 from processing; numProcessing=1 2023-07-15 11:55:37,753 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46827,1689422123904, splitWal=true, meta=true in 4.8340 sec 2023-07-15 11:55:37,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:37,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8bb3eb49d5c769961a9b0f7da157f1d1, NAME => 't2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:37,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:37,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:37,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:37,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:37,854 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:37,855 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1 2023-07-15 11:55:37,855 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1 2023-07-15 11:55:37,855 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8bb3eb49d5c769961a9b0f7da157f1d1 columnFamilyName cf1 2023-07-15 11:55:37,861 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1/b3a999ddc2d147879011cf288ac4fcc0 2023-07-15 11:55:37,861 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(310): Store=8bb3eb49d5c769961a9b0f7da157f1d1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:37,861 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:37,862 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/norep 2023-07-15 11:55:37,862 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/norep 2023-07-15 11:55:37,863 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8bb3eb49d5c769961a9b0f7da157f1d1 columnFamilyName norep 2023-07-15 11:55:37,864 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(310): Store=8bb3eb49d5c769961a9b0f7da157f1d1/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:37,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:37,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:37,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-15 11:55:37,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:37,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8bb3eb49d5c769961a9b0f7da157f1d1; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74232657, jitterRate=0.10615278780460358}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-15 11:55:37,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8bb3eb49d5c769961a9b0f7da157f1d1: 2023-07-15 11:55:37,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1., pid=26, masterSystemTime=1689422137848 2023-07-15 11:55:37,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:37,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:37,873 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=8bb3eb49d5c769961a9b0f7da157f1d1, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:37,874 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422137873"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422137873"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422137873"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422137873"}]},"ts":"1689422137873"} 2023-07-15 11:55:37,877 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-07-15 11:55:37,878 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure 8bb3eb49d5c769961a9b0f7da157f1d1, server=jenkins-hbase4.apache.org,33403,1689422132751 in 180 msec 2023-07-15 11:55:37,880 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=18 2023-07-15 11:55:37,880 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45535,1689422123929 after splitting done 2023-07-15 11:55:37,880 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN in 342 msec 2023-07-15 11:55:37,880 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45535,1689422123929 from processing; numProcessing=0 2023-07-15 11:55:37,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45535,1689422123929, splitWal=true, meta=false in 4.9430 sec 2023-07-15 11:55:38,050 INFO [jenkins-hbase4:33403Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:55:38,518 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-15 11:55:38,532 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-15 11:55:38,535 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-15 11:55:38,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.755sec 2023-07-15 11:55:38,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-15 11:55:38,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-15 11:55:38,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-15 11:55:38,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35607,1689422132725-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-15 11:55:38,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35607,1689422132725-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-15 11:55:38,539 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-15 11:55:38,576 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(139): Connect 0x505e097f to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:38,581 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@254317f8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:38,582 DEBUG [hconnection-0x336f042d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:38,584 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52050, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:38,587 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-15 11:55:38,588 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x505e097f to 127.0.0.1:57666 2023-07-15 11:55:38,588 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:38,590 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:35607 after: jenkins-hbase4.apache.org:35607 2023-07-15 11:55:38,591 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(139): Connect 0x61fa7a40 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:38,595 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2aff443e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:39,023 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-15 11:55:39,023 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-15 11:55:39,024 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-15 11:55:39,024 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-15 11:55:39,097 DEBUG [hconnection-0x2377498f-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:39,100 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52060, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:39,121 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689422139121 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:39,124 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): syncupReplication16894221391210x0, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:39,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689422139121-0x10168d630520022 connected Start Replication Server start 2023-07-15 11:55:39,127 INFO [Listener at localhost/46025] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:39,127 INFO [Listener at localhost/46025] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:39,139 INFO [Listener at localhost/46025] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-15 11:55:39,140 DEBUG [Listener at localhost/46025] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:55:39,150 INFO [Listener at localhost/46025] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-15 11:55:39,151 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,35805,1689422119867, jenkins-hbase4.apache.org,34419,1689422119824] other RSs: [] 2023-07-15 11:55:39,170 DEBUG [Listener at localhost/46025.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x1e71468b to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:39,176 DEBUG [Listener at localhost/46025.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@42858c46, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:39,176 INFO [Listener at localhost/46025.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:39,179 DEBUG [Listener at localhost/46025.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:39,180 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10168d630520024 connected 2023-07-15 11:55:39,180 INFO [Listener at localhost/46025.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=f0521d31-92f9-4caa-8fd4-4b1a00343aa2 to cluster=e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:42,628 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,35805,1689422119867/1's WALs to 1689422139127.syncuptool.replication.org,1234,1 2023-07-15 11:55:42,632 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(403): Removed empty jenkins-hbase4.apache.org,35805,1689422119867/1 2023-07-15 11:55:43,050 INFO [jenkins-hbase4:33403Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:55:43,205 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-15 11:55:45,993 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,34419,1689422119824/1's WALs to 1689422139127.syncuptool.replication.org,1234,1 2023-07-15 11:55:45,995 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276 with data PBUF\x08\xDFU 2023-07-15 11:55:45,996 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228 with data PBUF\x08\xFD\xAD\x01 2023-07-15 11:55:45,998 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,34419,1689422119824/1's WALs to 1689422139127.syncuptool.replication.org,1234,1 2023-07-15 11:55:48,000 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34419,1689422119824] 2023-07-15 11:55:48,000 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34419,1689422119824] 2023-07-15 11:55:48,015 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34419,1689422119824] 2023-07-15 11:55:48,015 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,34419,1689422119824, ReplicationSource: 1, currentBandwidth=0 2023-07-15 11:55:48,015 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:48,041 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824] zookeeper.ReadOnlyZKClient(139): Connect 0x3aa4b160 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:48,047 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5f299ea0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:48,047 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:48,050 INFO [jenkins-hbase4:33403Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:55:48,052 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:48,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10168d630520026 connected 2023-07-15 11:55:48,053 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,34419,1689422119824 (queues=2) is replicating from cluster=f0521d31-92f9-4caa-8fd4-4b1a00343aa2 to cluster=e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:48,053 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1 2023-07-15 11:55:48,057 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,34419,1689422119824, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-15 11:55:48,057 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0 2023-07-15 11:55:48,057 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1 2023-07-15 11:55:48,059 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,34419,1689422119824, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-15 11:55:48,060 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0 2023-07-15 11:55:48,066 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228 2023-07-15 11:55:48,066 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276 2023-07-15 11:55:48,066 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-15 11:55:48,066 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-15 11:55:48,066 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-15 11:55:48,066 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-15 11:55:48,068 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,34419,1689422119824 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/eee02c0767aa2be4d83f125ea78097e3/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276, lastWalPosition=15173, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-07-15 11:55:48,068 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,34419,1689422119824 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/89b547022cb2e9610cb6246270f3cbaf/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs/jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228, lastWalPosition=30605, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-07-15 11:55:48,070 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:48,070 INFO [SinkThread-1] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-07-15 11:55:48,075 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53890, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-15 11:55:48,077 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33403] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:55:48,077 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=33403] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:55:48,077 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33403] zookeeper.ReadOnlyZKClient(139): Connect 0x65599ce2 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:48,082 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33403] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@6b9e6e39, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:48,085 DEBUG [hconnection-0x6fa1f792-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:48,087 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53904, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-07-15 11:55:48,098 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33403] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:55:48,099 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34419,1689422119824] 2023-07-15 11:55:48,101 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=33403] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:55:48,102 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34419,1689422119824] 2023-07-15 11:55:48,104 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0.1689422121276] 2023-07-15 11:55:48,105 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1.1689422122228] 2023-07-15 11:55:48,106 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,34419,1689422119824 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-15 11:55:48,106 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:48,108 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,34419,1689422119824 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-15 11:55:48,108 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:48,207 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:48,209 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689422139121-0x10168d630520022, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689422139127.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:48,209 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-0,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,34419,1689422119824 with the following stats: Total replicated edits: 2, current progress: 2023-07-15 11:55:48,309 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,34419,1689422119824 2023-07-15 11:55:48,310 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] zookeeper.ZKUtil(243): syncupReplication1689422139121-0x10168d630520022, quorum=127.0.0.1:57666, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689422139127.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34419,1689422119824 because node does not exist (not an error) 2023-07-15 11:55:48,310 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824.replicationSource.shipperjenkins-hbase4.apache.org%2C34419%2C1689422119824.jenkins-hbase4.apache.org%2C34419%2C1689422119824.regiongroup-1,1-jenkins-hbase4.apache.org,34419,1689422119824] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,34419,1689422119824 with the following stats: Total replicated edits: 2, current progress: 2023-07-15 11:55:49,153 INFO [Listener at localhost/46025] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-15 11:55:49,256 DEBUG [Listener at localhost/46025.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520024, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:49,256 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1e71468b to 127.0.0.1:57666 2023-07-15 11:55:49,258 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:49,459 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689422139121-0x10168d630520022, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:49,459 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689422139121-0x10168d630520022, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:49,472 INFO [Listener at localhost/46025] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-07-15 11:55:49,473 DEBUG [Listener at localhost/46025] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-07-15 11:55:49,474 INFO [Listener at localhost/46025] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-15 11:55:49,474 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:49,474 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:49,475 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:49,475 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:49,475 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:49,475 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:49,477 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42779 2023-07-15 11:55:49,477 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:49,478 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:49,480 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=master:42779 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:49,482 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:427790x0, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:49,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:42779-0x10168d630520028 connected 2023-07-15 11:55:49,486 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-15 11:55:49,486 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-15 11:55:49,486 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-15 11:55:49,487 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42779 2023-07-15 11:55:49,487 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42779 2023-07-15 11:55:49,487 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42779 2023-07-15 11:55:49,488 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42779 2023-07-15 11:55:49,488 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42779 2023-07-15 11:55:49,488 INFO [Listener at localhost/46025] master.HMaster(444): hbase.rootdir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1, hbase.cluster.distributed=false 2023-07-15 11:55:49,489 DEBUG [pool-558-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-07-15 11:55:49,499 INFO [Listener at localhost/46025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-15 11:55:49,499 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:49,500 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:49,500 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:49,500 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:49,500 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:49,500 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:49,502 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46571 2023-07-15 11:55:49,502 INFO [Listener at localhost/46025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-15 11:55:49,503 DEBUG [Listener at localhost/46025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-15 11:55:49,504 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:49,504 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:49,505 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46571 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:49,507 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:465710x0, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:49,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46571-0x10168d630520029 connected 2023-07-15 11:55:49,510 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-15 11:55:49,510 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-15 11:55:49,511 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-15 11:55:49,512 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46571 2023-07-15 11:55:49,512 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46571 2023-07-15 11:55:49,515 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46571 2023-07-15 11:55:49,515 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46571 2023-07-15 11:55:49,515 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46571 2023-07-15 11:55:49,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:49,519 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-15 11:55:49,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:49,521 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-15 11:55:49,521 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-15 11:55:49,521 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:49,523 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-15 11:55:49,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-15 11:55:49,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,42779,1689422149474 from backup master directory 2023-07-15 11:55:49,525 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:49,525 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-15 11:55:49,525 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:49,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:49,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:49,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x55854855 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:49,565 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@68633219, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:49,565 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:49,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-15 11:55:49,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:49,568 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:49,573 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,38223,1689422118510 to hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,38223,1689422118510-dead as it is dead 2023-07-15 11:55:49,573 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,38223,1689422118510-dead/jenkins-hbase4.apache.org%2C38223%2C1689422118510.jenkins-hbase4.apache.org%2C38223%2C1689422118510.regiongroup-0.1689422120328 2023-07-15 11:55:49,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,38223,1689422118510-dead/jenkins-hbase4.apache.org%2C38223%2C1689422118510.jenkins-hbase4.apache.org%2C38223%2C1689422118510.regiongroup-0.1689422120328 after 1ms 2023-07-15 11:55:49,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,38223,1689422118510-dead/jenkins-hbase4.apache.org%2C38223%2C1689422118510.jenkins-hbase4.apache.org%2C38223%2C1689422118510.regiongroup-0.1689422120328 to hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38223%2C1689422118510.jenkins-hbase4.apache.org%2C38223%2C1689422118510.regiongroup-0.1689422120328 2023-07-15 11:55:49,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,38223,1689422118510-dead 2023-07-15 11:55:49,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:49,576 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:49,577 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42779%2C1689422149474.jenkins-hbase4.apache.org%2C42779%2C1689422149474.regiongroup-0, suffix=, logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,42779,1689422149474, archiveDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/oldWALs, maxLogs=10 2023-07-15 11:55:49,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,42779,1689422149474/jenkins-hbase4.apache.org%2C42779%2C1689422149474.jenkins-hbase4.apache.org%2C42779%2C1689422149474.regiongroup-0.1689422149578 2023-07-15 11:55:49,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37061,DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8,DISK], DatanodeInfoWithStorage[127.0.0.1:42115,DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e,DISK]] 2023-07-15 11:55:49,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:49,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:49,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:49,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:49,585 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:49,586 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-15 11:55:49,586 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-15 11:55:49,592 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ee49164dbdb445559382832f382eae1a 2023-07-15 11:55:49,592 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:49,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-15 11:55:49,593 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38223%2C1689422118510.jenkins-hbase4.apache.org%2C38223%2C1689422118510.regiongroup-0.1689422120328 2023-07-15 11:55:49,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38223%2C1689422118510.jenkins-hbase4.apache.org%2C38223%2C1689422118510.regiongroup-0.1689422120328 2023-07-15 11:55:49,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38223%2C1689422118510.jenkins-hbase4.apache.org%2C38223%2C1689422118510.regiongroup-0.1689422120328 2023-07-15 11:55:49,601 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:49,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-07-15 11:55:49,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72857739, jitterRate=0.08566491305828094}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-15 11:55:49,608 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:49,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-15 11:55:49,610 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-15 11:55:49,610 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-15 11:55:49,610 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-15 11:55:49,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-15 11:55:49,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-15 11:55:49,616 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-15 11:55:49,616 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-15 11:55:49,616 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-15 11:55:49,616 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-15 11:55:49,617 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-15 11:55:49,617 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-15 11:55:49,617 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-15 11:55:49,617 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-15 11:55:49,619 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-15 11:55:49,619 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34419,1689422119824, table=hbase:meta, region=1588230740 2023-07-15 11:55:49,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-07-15 11:55:49,622 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,35805,1689422119867 already deleted, retry=false 2023-07-15 11:55:49,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,35805,1689422119867 on jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:49,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,35805,1689422119867, splitWal=true, meta=false 2023-07-15 11:55:49,625 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,35805,1689422119867 (carryingMeta=false) jenkins-hbase4.apache.org,35805,1689422119867/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@1b28cd43[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-15 11:55:49,626 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,34419,1689422119824 already deleted, retry=false 2023-07-15 11:55:49,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34419,1689422119824 on jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:49,627 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34419,1689422119824, splitWal=true, meta=true 2023-07-15 11:55:49,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase4.apache.org,34419,1689422119824 (carryingMeta=true) jenkins-hbase4.apache.org,34419,1689422119824/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@5ecd00d2[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-15 11:55:49,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-15 11:55:49,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-15 11:55:49,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-15 11:55:49,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-15 11:55:49,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-15 11:55:49,640 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-15 11:55:49,640 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-15 11:55:49,643 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-15 11:55:49,645 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-15 11:55:49,645 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-15 11:55:49,645 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:49,645 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,42779,1689422149474, sessionid=0x10168d630520028, setting cluster-up flag (Was=false) 2023-07-15 11:55:49,652 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-15 11:55:49,652 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:49,655 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-15 11:55:49,656 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:49,656 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/.hbase-snapshot/.tmp 2023-07-15 11:55:49,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:49,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:49,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:49,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:49,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-15 11:55:49,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:49,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689422179665 2023-07-15 11:55:49,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-15 11:55:49,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-15 11:55:49,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-15 11:55:49,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-15 11:55:49,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-15 11:55:49,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-15 11:55:49,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:49,666 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34419,1689422119824; numProcessing=1 2023-07-15 11:55:49,666 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35805,1689422119867; numProcessing=2 2023-07-15 11:55:49,666 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34419,1689422119824, splitWal=true, meta=true 2023-07-15 11:55:49,666 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,35805,1689422119867, splitWal=true, meta=false 2023-07-15 11:55:49,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-15 11:55:49,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-15 11:55:49,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-15 11:55:49,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-15 11:55:49,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-15 11:55:49,668 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34419,1689422119824, splitWal=true, meta=true, isMeta: true 2023-07-15 11:55:49,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422149668,5,FailOnTimeoutGroup] 2023-07-15 11:55:49,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422149671,5,FailOnTimeoutGroup] 2023-07-15 11:55:49,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:49,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-15 11:55:49,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:49,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:49,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689422149673, completionTime=-1 2023-07-15 11:55:49,673 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-15 11:55:49,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-15 11:55:49,673 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824-splitting 2023-07-15 11:55:49,674 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824-splitting dir is empty, no logs to split. 2023-07-15 11:55:49,674 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34419,1689422119824 WAL count=0, meta=true 2023-07-15 11:55:49,677 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824-splitting dir is empty, no logs to split. 2023-07-15 11:55:49,677 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34419,1689422119824 WAL count=0, meta=true 2023-07-15 11:55:49,677 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34419,1689422119824 WAL splitting is done? wals=0, meta=true 2023-07-15 11:55:49,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-15 11:55:49,680 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-15 11:55:49,682 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:55:49,717 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(951): ClusterId : f0521d31-92f9-4caa-8fd4-4b1a00343aa2 2023-07-15 11:55:49,717 DEBUG [RS:0;jenkins-hbase4:46571] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-15 11:55:49,719 DEBUG [RS:0;jenkins-hbase4:46571] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-15 11:55:49,719 DEBUG [RS:0;jenkins-hbase4:46571] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-15 11:55:49,722 DEBUG [RS:0;jenkins-hbase4:46571] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-15 11:55:49,724 DEBUG [RS:0;jenkins-hbase4:46571] zookeeper.ReadOnlyZKClient(139): Connect 0x0d1153b8 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:49,728 DEBUG [RS:0;jenkins-hbase4:46571] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3d6bc6a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:49,728 DEBUG [RS:0;jenkins-hbase4:46571] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e938c8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:49,740 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46571 2023-07-15 11:55:49,740 INFO [RS:0;jenkins-hbase4:46571] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-15 11:55:49,740 INFO [RS:0;jenkins-hbase4:46571] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-15 11:55:49,740 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1022): About to register with Master. 2023-07-15 11:55:49,741 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,42779,1689422149474 with isa=jenkins-hbase4.apache.org/172.31.14.131:46571, startcode=1689422149499 2023-07-15 11:55:49,741 DEBUG [RS:0;jenkins-hbase4:46571] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-15 11:55:49,748 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36505, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-15 11:55:49,749 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=42779] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:49,750 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1 2023-07-15 11:55:49,750 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:44973 2023-07-15 11:55:49,750 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-15 11:55:49,752 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-15 11:55:49,752 DEBUG [RS:0;jenkins-hbase4:46571] zookeeper.ZKUtil(162): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:49,753 WARN [RS:0;jenkins-hbase4:46571] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:49,753 INFO [RS:0;jenkins-hbase4:46571] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:49,753 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46571,1689422149499] 2023-07-15 11:55:49,753 INFO [RS:0;jenkins-hbase4:46571] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:49,754 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1948): logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:49,770 INFO [RS:0;jenkins-hbase4:46571] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-15 11:55:49,771 DEBUG [RS:0;jenkins-hbase4:46571] zookeeper.ZKUtil(162): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:49,772 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:55:49,772 INFO [RS:0;jenkins-hbase4:46571] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-15 11:55:49,773 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=100ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-15 11:55:49,774 INFO [RS:0;jenkins-hbase4:46571] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-15 11:55:49,779 INFO [RS:0;jenkins-hbase4:46571] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-15 11:55:49,779 INFO [RS:0;jenkins-hbase4:46571] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:49,779 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-15 11:55:49,781 INFO [RS:0;jenkins-hbase4:46571] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:49,781 DEBUG [RS:0;jenkins-hbase4:46571] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,781 DEBUG [RS:0;jenkins-hbase4:46571] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,781 DEBUG [RS:0;jenkins-hbase4:46571] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,781 DEBUG [RS:0;jenkins-hbase4:46571] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,781 DEBUG [RS:0;jenkins-hbase4:46571] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,781 DEBUG [RS:0;jenkins-hbase4:46571] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:49,781 DEBUG [RS:0;jenkins-hbase4:46571] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,782 DEBUG [RS:0;jenkins-hbase4:46571] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,782 DEBUG [RS:0;jenkins-hbase4:46571] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,782 DEBUG [RS:0;jenkins-hbase4:46571] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:49,783 INFO [RS:0;jenkins-hbase4:46571] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:49,783 INFO [RS:0;jenkins-hbase4:46571] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:49,783 INFO [RS:0;jenkins-hbase4:46571] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:49,795 INFO [RS:0;jenkins-hbase4:46571] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-15 11:55:49,795 INFO [RS:0;jenkins-hbase4:46571] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46571,1689422149499-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:49,807 INFO [RS:0;jenkins-hbase4:46571] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-15 11:55:49,808 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1689422139127.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase4.apache.org,46571,1689422149499] 2023-07-15 11:55:49,819 INFO [RS:0;jenkins-hbase4:46571] regionserver.Replication(203): jenkins-hbase4.apache.org,46571,1689422149499 started 2023-07-15 11:55:49,819 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46571,1689422149499, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46571, sessionid=0x10168d630520029 2023-07-15 11:55:49,819 DEBUG [RS:0;jenkins-hbase4:46571] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-15 11:55:49,819 DEBUG [RS:0;jenkins-hbase4:46571] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:49,819 DEBUG [RS:0;jenkins-hbase4:46571] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46571,1689422149499' 2023-07-15 11:55:49,819 DEBUG [RS:0;jenkins-hbase4:46571] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-15 11:55:49,820 DEBUG [RS:0;jenkins-hbase4:46571] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-15 11:55:49,820 DEBUG [RS:0;jenkins-hbase4:46571] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-15 11:55:49,820 DEBUG [RS:0;jenkins-hbase4:46571] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-15 11:55:49,820 DEBUG [RS:0;jenkins-hbase4:46571] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:49,820 DEBUG [RS:0;jenkins-hbase4:46571] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46571,1689422149499' 2023-07-15 11:55:49,820 DEBUG [RS:0;jenkins-hbase4:46571] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-15 11:55:49,820 DEBUG [RS:0;jenkins-hbase4:46571] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-15 11:55:49,821 DEBUG [RS:0;jenkins-hbase4:46571] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-15 11:55:49,821 INFO [RS:0;jenkins-hbase4:46571] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-15 11:55:49,821 INFO [RS:0;jenkins-hbase4:46571] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-15 11:55:49,827 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x34901999 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:49,831 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@26381476, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:49,831 INFO [RS:0;jenkins-hbase4:46571.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:49,832 DEBUG [jenkins-hbase4:42779] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-15 11:55:49,833 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46571,1689422149499, state=OPENING 2023-07-15 11:55:49,836 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:49,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10168d63052002d connected 2023-07-15 11:55:49,837 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-15 11:55:49,837 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-15 11:55:49,837 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46571,1689422149499}] 2023-07-15 11:55:49,837 INFO [RS:0;jenkins-hbase4:46571.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=f0521d31-92f9-4caa-8fd4-4b1a00343aa2 to cluster=e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:49,921 INFO [RS:0;jenkins-hbase4:46571] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:49,923 INFO [RS:0;jenkins-hbase4:46571] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0, suffix=, logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499, archiveDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs, maxLogs=10 2023-07-15 11:55:49,940 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0 for peer 1 2023-07-15 11:55:49,940 INFO [RS:0;jenkins-hbase4:46571] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0.1689422149924 2023-07-15 11:55:49,940 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0 2023-07-15 11:55:49,940 INFO [RS:0;jenkins-hbase4:46571] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-15 11:55:49,940 DEBUG [RS:0;jenkins-hbase4:46571] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42115,DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e,DISK], DatanodeInfoWithStorage[127.0.0.1:37061,DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8,DISK]] 2023-07-15 11:55:49,940 INFO [RS:0;jenkins-hbase4:46571.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0 2023-07-15 11:55:49,946 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:49,995 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:49,995 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:49,997 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32978, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-15 11:55:50,000 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-15 11:55:50,000 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:50,000 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:50,000 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:50,002 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46571%2C1689422149499.meta, suffix=.meta, logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499, archiveDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs, maxLogs=10 2023-07-15 11:55:50,007 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.meta.1689422150002.meta 2023-07-15 11:55:50,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37061,DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8,DISK], DatanodeInfoWithStorage[127.0.0.1:42115,DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e,DISK]] 2023-07-15 11:55:50,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:50,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-15 11:55:50,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-15 11:55:50,007 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-15 11:55:50,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-15 11:55:50,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:50,008 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-15 11:55:50,008 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-15 11:55:50,009 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-15 11:55:50,010 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/info 2023-07-15 11:55:50,010 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/info 2023-07-15 11:55:50,010 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-15 11:55:50,016 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/info/163103135f7e4a54a2604cb8c7667c56 2023-07-15 11:55:50,016 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:50,016 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-15 11:55:50,017 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:50,017 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:50,017 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-15 11:55:50,022 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/rep_barrier/aa1178b203d84561adbcd93c00f5a776 2023-07-15 11:55:50,023 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:50,023 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-15 11:55:50,023 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/table 2023-07-15 11:55:50,024 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/table 2023-07-15 11:55:50,024 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-15 11:55:50,029 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/table/6e4f2bca2c9b4940b4ad448e0fb75048 2023-07-15 11:55:50,029 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:50,030 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740 2023-07-15 11:55:50,031 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740 2023-07-15 11:55:50,033 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-15 11:55:50,035 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-15 11:55:50,035 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66047383, jitterRate=-0.015817299485206604}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-15 11:55:50,035 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-15 11:55:50,041 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1689422149995 2023-07-15 11:55:50,046 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-15 11:55:50,047 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-15 11:55:50,047 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46571,1689422149499, state=OPEN 2023-07-15 11:55:50,049 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-15 11:55:50,049 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-15 11:55:50,052 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-07-15 11:55:50,052 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46571,1689422149499 in 212 msec 2023-07-15 11:55:50,053 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:50,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-15 11:55:50,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 374 msec 2023-07-15 11:55:50,158 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:50,263 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:50,368 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:50,473 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:50,578 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:50,683 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:50,788 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:50,893 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:50,999 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:51,104 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:51,210 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:51,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1603ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-15 11:55:51,315 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:51,421 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:51,527 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:51,632 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:51,738 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:51,844 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:51,949 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:52,056 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:52,162 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:52,267 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:52,372 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:52,477 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:52,583 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:52,688 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:52,778 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3105ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-15 11:55:52,793 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:52,898 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:53,003 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:53,050 INFO [jenkins-hbase4:33403Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:55:53,050 INFO [jenkins-hbase4:33403Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-15 11:55:53,108 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:53,214 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:53,319 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:53,424 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:53,530 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:53,635 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:53,712 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-15 11:55:53,744 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:53,849 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:53,954 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:54,060 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:54,165 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:54,180 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-15 11:55:54,180 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-15 11:55:54,184 DEBUG [hconnection-0x33ff84d9-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:54,186 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42386, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:54,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=f1407b2bc354da8a7c08c95abb9ab378, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34419,1689422119824, regionLocation=jenkins-hbase4.apache.org,34419,1689422119824, openSeqNum=2 2023-07-15 11:55:54,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=eee02c0767aa2be4d83f125ea78097e3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34419,1689422119824, regionLocation=jenkins-hbase4.apache.org,34419,1689422119824, openSeqNum=2 2023-07-15 11:55:54,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=89b547022cb2e9610cb6246270f3cbaf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34419,1689422119824, regionLocation=jenkins-hbase4.apache.org,34419,1689422119824, openSeqNum=2 2023-07-15 11:55:54,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-15 11:55:54,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689422214193 2023-07-15 11:55:54,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689422274193 2023-07-15 11:55:54,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 12 msec 2023-07-15 11:55:54,209 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34419,1689422119824 had 4 regions 2023-07-15 11:55:54,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42779,1689422149474-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:54,210 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,35805,1689422119867 had 0 regions 2023-07-15 11:55:54,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42779,1689422149474-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:54,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42779,1689422149474-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:54,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:42779, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:54,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:54,210 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. is NOT online; state={f1407b2bc354da8a7c08c95abb9ab378 state=OPEN, ts=1689422154192, server=jenkins-hbase4.apache.org,34419,1689422119824}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-15 11:55:54,211 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34419,1689422119824, splitWal=true, meta=true, isMeta: false 2023-07-15 11:55:54,211 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,35805,1689422119867, splitWal=true, meta=false, isMeta: false 2023-07-15 11:55:54,214 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,34419,1689422119824/hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378., unknown_server=jenkins-hbase4.apache.org,34419,1689422119824/t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3., unknown_server=jenkins-hbase4.apache.org,34419,1689422119824/t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:54,214 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824-splitting dir is empty, no logs to split. 2023-07-15 11:55:54,214 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,34419,1689422119824 WAL count=0, meta=false 2023-07-15 11:55:54,214 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867-splitting 2023-07-15 11:55:54,215 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867-splitting dir is empty, no logs to split. 2023-07-15 11:55:54,215 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,35805,1689422119867 WAL count=0, meta=false 2023-07-15 11:55:54,217 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,34419,1689422119824-splitting dir is empty, no logs to split. 2023-07-15 11:55:54,217 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,34419,1689422119824 WAL count=0, meta=false 2023-07-15 11:55:54,217 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34419,1689422119824 WAL splitting is done? wals=0, meta=false 2023-07-15 11:55:54,217 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867-splitting dir is empty, no logs to split. 2023-07-15 11:55:54,218 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,35805,1689422119867 WAL count=0, meta=false 2023-07-15 11:55:54,218 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,35805,1689422119867 WAL splitting is done? wals=0, meta=false 2023-07-15 11:55:54,219 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f1407b2bc354da8a7c08c95abb9ab378, ASSIGN}, {pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=eee02c0767aa2be4d83f125ea78097e3, ASSIGN}, {pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=89b547022cb2e9610cb6246270f3cbaf, ASSIGN}] 2023-07-15 11:55:54,220 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,35805,1689422119867 failed, ignore...File hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,35805,1689422119867-splitting does not exist. 2023-07-15 11:55:54,220 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f1407b2bc354da8a7c08c95abb9ab378, ASSIGN 2023-07-15 11:55:54,220 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=89b547022cb2e9610cb6246270f3cbaf, ASSIGN 2023-07-15 11:55:54,220 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=eee02c0767aa2be4d83f125ea78097e3, ASSIGN 2023-07-15 11:55:54,221 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=f1407b2bc354da8a7c08c95abb9ab378, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:55:54,221 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=89b547022cb2e9610cb6246270f3cbaf, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:55:54,221 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=eee02c0767aa2be4d83f125ea78097e3, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:55:54,221 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,35805,1689422119867 after splitting done 2023-07-15 11:55:54,221 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,35805,1689422119867 from processing; numProcessing=1 2023-07-15 11:55:54,223 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,35805,1689422119867, splitWal=true, meta=false in 4.6000 sec 2023-07-15 11:55:54,270 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:54,371 DEBUG [jenkins-hbase4:42779] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-15 11:55:54,375 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=89b547022cb2e9610cb6246270f3cbaf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:54,375 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=f1407b2bc354da8a7c08c95abb9ab378, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:54,375 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422154374"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422154374"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422154374"}]},"ts":"1689422154374"} 2023-07-15 11:55:54,375 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422154374"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422154374"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422154374"}]},"ts":"1689422154374"} 2023-07-15 11:55:54,375 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=eee02c0767aa2be4d83f125ea78097e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:54,375 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422154374"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422154374"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422154374"}]},"ts":"1689422154374"} 2023-07-15 11:55:54,378 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 89b547022cb2e9610cb6246270f3cbaf, server=jenkins-hbase4.apache.org,46571,1689422149499}] 2023-07-15 11:55:54,379 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:54,383 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=22, state=RUNNABLE; OpenRegionProcedure f1407b2bc354da8a7c08c95abb9ab378, server=jenkins-hbase4.apache.org,46571,1689422149499}] 2023-07-15 11:55:54,384 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=23, state=RUNNABLE; OpenRegionProcedure eee02c0767aa2be4d83f125ea78097e3, server=jenkins-hbase4.apache.org,46571,1689422149499}] 2023-07-15 11:55:54,484 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:54,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:54,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:54,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1, suffix=, logDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499, archiveDir=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs, maxLogs=10 2023-07-15 11:55:54,539 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:54,539 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f1407b2bc354da8a7c08c95abb9ab378, NAME => 'hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:54,539 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:54,539 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:54,539 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:54,539 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:54,541 INFO [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:54,543 DEBUG [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/info 2023-07-15 11:55:54,543 DEBUG [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/info 2023-07-15 11:55:54,543 INFO [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1407b2bc354da8a7c08c95abb9ab378 columnFamilyName info 2023-07-15 11:55:54,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1 for peer 1 2023-07-15 11:55:54,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1.1689422154536 2023-07-15 11:55:54,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1 2023-07-15 11:55:54,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-15 11:55:54,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42115,DS-0842addc-f575-4fe4-a47d-dd1a16e44b1e,DISK], DatanodeInfoWithStorage[127.0.0.1:37061,DS-2f80c379-50f4-4817-8dfa-fcd9a824eaf8,DISK]] 2023-07-15 11:55:54,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1 2023-07-15 11:55:54,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eee02c0767aa2be4d83f125ea78097e3, NAME => 't1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:54,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:54,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:54,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:54,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:54,554 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:54,555 DEBUG [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/cf1 2023-07-15 11:55:54,555 DEBUG [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/cf1 2023-07-15 11:55:54,556 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eee02c0767aa2be4d83f125ea78097e3 columnFamilyName cf1 2023-07-15 11:55:54,558 DEBUG [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] regionserver.HStore(539): loaded hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/info/7c1a732702e24f1f847a91dd73b1f9e1 2023-07-15 11:55:54,561 INFO [StoreOpener-f1407b2bc354da8a7c08c95abb9ab378-1] regionserver.HStore(310): Store=f1407b2bc354da8a7c08c95abb9ab378/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:54,562 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:54,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-15 11:55:54,563 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:54,567 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 983cbb2db3c5484487fba3518cd32e6b 2023-07-15 11:55:54,567 DEBUG [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] regionserver.HStore(539): loaded hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/cf1/983cbb2db3c5484487fba3518cd32e6b 2023-07-15 11:55:54,567 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:54,567 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] regionserver.HStore(310): Store=eee02c0767aa2be4d83f125ea78097e3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:54,567 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:54,568 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f1407b2bc354da8a7c08c95abb9ab378; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73534191, jitterRate=0.09574483335018158}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-15 11:55:54,568 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f1407b2bc354da8a7c08c95abb9ab378: 2023-07-15 11:55:54,568 DEBUG [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/norep 2023-07-15 11:55:54,568 DEBUG [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/norep 2023-07-15 11:55:54,570 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eee02c0767aa2be4d83f125ea78097e3 columnFamilyName norep 2023-07-15 11:55:54,571 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378., pid=26, masterSystemTime=1689422154530 2023-07-15 11:55:54,574 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:54,574 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:54,575 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=f1407b2bc354da8a7c08c95abb9ab378, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:54,575 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422154575"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422154575"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422154575"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422154575"}]},"ts":"1689422154575"} 2023-07-15 11:55:54,578 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65c44ba5926d4820bfdf72628e0df5c8 2023-07-15 11:55:54,578 DEBUG [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] regionserver.HStore(539): loaded hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/norep/65c44ba5926d4820bfdf72628e0df5c8 2023-07-15 11:55:54,578 INFO [StoreOpener-eee02c0767aa2be4d83f125ea78097e3-1] regionserver.HStore(310): Store=eee02c0767aa2be4d83f125ea78097e3/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:54,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:54,579 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=22 2023-07-15 11:55:54,579 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=22, state=SUCCESS; OpenRegionProcedure f1407b2bc354da8a7c08c95abb9ab378, server=jenkins-hbase4.apache.org,46571,1689422149499 in 194 msec 2023-07-15 11:55:54,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:54,580 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=f1407b2bc354da8a7c08c95abb9ab378, ASSIGN in 361 msec 2023-07-15 11:55:54,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-15 11:55:54,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:54,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened eee02c0767aa2be4d83f125ea78097e3; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71586779, jitterRate=0.06672613322734833}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-15 11:55:54,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for eee02c0767aa2be4d83f125ea78097e3: 2023-07-15 11:55:54,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3., pid=27, masterSystemTime=1689422154530 2023-07-15 11:55:54,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:54,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:54,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:54,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 89b547022cb2e9610cb6246270f3cbaf, NAME => 't2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:54,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:54,589 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=eee02c0767aa2be4d83f125ea78097e3, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:54,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:54,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:54,589 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422154589"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422154589"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422154589"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422154589"}]},"ts":"1689422154589"} 2023-07-15 11:55:54,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:54,591 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:54,592 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:54,592 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-15 11:55:54,592 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:54,592 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0.1689422149924, lastWalPosition=422, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-15 11:55:54,592 DEBUG [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/cf1 2023-07-15 11:55:54,592 DEBUG [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/cf1 2023-07-15 11:55:54,593 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 89b547022cb2e9610cb6246270f3cbaf columnFamilyName cf1 2023-07-15 11:55:54,594 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=23 2023-07-15 11:55:54,594 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=23, state=SUCCESS; OpenRegionProcedure eee02c0767aa2be4d83f125ea78097e3, server=jenkins-hbase4.apache.org,46571,1689422149499 in 207 msec 2023-07-15 11:55:54,595 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=eee02c0767aa2be4d83f125ea78097e3, ASSIGN in 376 msec 2023-07-15 11:55:54,598 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3d44f5295edd462dbbf3383b29306943 2023-07-15 11:55:54,598 DEBUG [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] regionserver.HStore(539): loaded hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/cf1/3d44f5295edd462dbbf3383b29306943 2023-07-15 11:55:54,598 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] regionserver.HStore(310): Store=89b547022cb2e9610cb6246270f3cbaf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:54,598 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:54,599 DEBUG [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/norep 2023-07-15 11:55:54,599 DEBUG [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/norep 2023-07-15 11:55:54,600 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 89b547022cb2e9610cb6246270f3cbaf columnFamilyName norep 2023-07-15 11:55:54,605 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68d8ade59a834b269f027ba67c4f7b60 2023-07-15 11:55:54,605 DEBUG [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] regionserver.HStore(539): loaded hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/norep/68d8ade59a834b269f027ba67c4f7b60 2023-07-15 11:55:54,605 INFO [StoreOpener-89b547022cb2e9610cb6246270f3cbaf-1] regionserver.HStore(310): Store=89b547022cb2e9610cb6246270f3cbaf/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:54,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:54,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:54,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-15 11:55:54,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:54,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 89b547022cb2e9610cb6246270f3cbaf; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66553329, jitterRate=-0.008278116583824158}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-15 11:55:54,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 89b547022cb2e9610cb6246270f3cbaf: 2023-07-15 11:55:54,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf., pid=25, masterSystemTime=1689422154530 2023-07-15 11:55:54,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:54,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:54,615 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=89b547022cb2e9610cb6246270f3cbaf, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:54,615 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422154615"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422154615"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422154615"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422154615"}]},"ts":"1689422154615"} 2023-07-15 11:55:54,619 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-07-15 11:55:54,619 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 89b547022cb2e9610cb6246270f3cbaf, server=jenkins-hbase4.apache.org,46571,1689422149499 in 239 msec 2023-07-15 11:55:54,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=19 2023-07-15 11:55:54,621 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34419,1689422119824 after splitting done 2023-07-15 11:55:54,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=89b547022cb2e9610cb6246270f3cbaf, ASSIGN in 401 msec 2023-07-15 11:55:54,621 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34419,1689422119824 from processing; numProcessing=0 2023-07-15 11:55:54,622 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34419,1689422119824, splitWal=true, meta=true in 4.9940 sec 2023-07-15 11:55:54,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:54,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-15 11:55:54,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:54,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1.1689422154536, lastWalPosition=823, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-15 11:55:54,697 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:54,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:54,801 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:54,819 INFO [jenkins-hbase4:46571Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0]: currently replicating from: hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0.1689422149924 at position: 422 walGroup [jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1]: currently replicating from: hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1.1689422154536 at position: 823 2023-07-15 11:55:54,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:54,906 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:54,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:55,010 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:55,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:55,115 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:55,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:55,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-07-15 11:55:55,221 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:55,225 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-15 11:55:55,227 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-15 11:55:55,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.702sec 2023-07-15 11:55:55,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-15 11:55:55,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-15 11:55:55,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-15 11:55:55,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42779,1689422149474-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-15 11:55:55,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42779,1689422149474-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-15 11:55:55,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-15 11:55:55,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:55,321 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(139): Connect 0x7f898095 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:55,326 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11ec8647, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:55,327 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:55,328 DEBUG [hconnection-0x1bb794cd-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:55,331 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42390, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:55,336 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-15 11:55:55,337 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7f898095 to 127.0.0.1:57666 2023-07-15 11:55:55,337 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:55,338 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:42779 after: jenkins-hbase4.apache.org:42779 2023-07-15 11:55:55,339 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(139): Connect 0x49f409d4 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:55,343 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6ec2fddb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:55,343 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x61fa7a40 to 127.0.0.1:57666 2023-07-15 11:55:55,343 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:55,343 DEBUG [Listener at localhost/46025] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-15 11:55:55,343 DEBUG [Listener at localhost/46025] util.JVMClusterUtil(257): Found active master hash=2096844401, stopped=false 2023-07-15 11:55:55,343 INFO [Listener at localhost/46025] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:55,346 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-15 11:55:55,346 INFO [Listener at localhost/46025] procedure2.ProcedureExecutor(629): Stopping 2023-07-15 11:55:55,346 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:55,346 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-15 11:55:55,347 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x100c031b to 127.0.0.1:57666 2023-07-15 11:55:55,348 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:55,348 INFO [Listener at localhost/46025] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33403,1689422132751' ***** 2023-07-15 11:55:55,348 INFO [Listener at localhost/46025] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-15 11:55:55,348 INFO [RS:0;jenkins-hbase4:33403] regionserver.HeapMemoryManager(220): Stopping 2023-07-15 11:55:55,348 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:55,349 INFO [RS:0;jenkins-hbase4:33403] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-15 11:55:55,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:55,349 INFO [RS:0;jenkins-hbase4:33403] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-15 11:55:55,349 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-15 11:55:55,349 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(3305): Received CLOSE for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:55:55,350 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(3305): Received CLOSE for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:55:55,350 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(3305): Received CLOSE for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:55,350 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:55,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ed49ae2049e592f2f5d8017bfcee4d2f, disabling compactions & flushes 2023-07-15 11:55:55,351 DEBUG [RS:0;jenkins-hbase4:33403] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5651b708 to 127.0.0.1:57666 2023-07-15 11:55:55,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:55,351 DEBUG [RS:0;jenkins-hbase4:33403] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:55,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:55,351 INFO [RS:0;jenkins-hbase4:33403] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-15 11:55:55,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. after waiting 0 ms 2023-07-15 11:55:55,351 INFO [RS:0;jenkins-hbase4:33403] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-15 11:55:55,351 INFO [RS:0;jenkins-hbase4:33403] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-15 11:55:55,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:55,351 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-15 11:55:55,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ed49ae2049e592f2f5d8017bfcee4d2f 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-07-15 11:55:55,351 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-15 11:55:55,351 DEBUG [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1478): Online Regions={ed49ae2049e592f2f5d8017bfcee4d2f=t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f., 1588230740=hbase:meta,,1.1588230740, 8bb3eb49d5c769961a9b0f7da157f1d1=t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1., faf8bf9c0e6d77a0bdf0f3a70ab43734=hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.} 2023-07-15 11:55:55,352 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-15 11:55:55,352 DEBUG [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1504): Waiting on 1588230740, 8bb3eb49d5c769961a9b0f7da157f1d1, ed49ae2049e592f2f5d8017bfcee4d2f, faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:55:55,352 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-15 11:55:55,352 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-15 11:55:55,352 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-15 11:55:55,352 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-15 11:55:55,353 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-15 11:55:55,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/.tmp/cf1/451bdab9c3e24511a03375f24cd75048 2023-07-15 11:55:55,372 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/.tmp/info/6c9410a40e3240ccadef6c95ab632e52 2023-07-15 11:55:55,377 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 451bdab9c3e24511a03375f24cd75048 2023-07-15 11:55:55,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/.tmp/cf1/451bdab9c3e24511a03375f24cd75048 as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1/451bdab9c3e24511a03375f24cd75048 2023-07-15 11:55:55,379 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/.tmp/info/6c9410a40e3240ccadef6c95ab632e52 as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info/6c9410a40e3240ccadef6c95ab632e52 2023-07-15 11:55:55,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 451bdab9c3e24511a03375f24cd75048 2023-07-15 11:55:55,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1/451bdab9c3e24511a03375f24cd75048, entries=50, sequenceid=14, filesize=6.4 K 2023-07-15 11:55:55,385 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for ed49ae2049e592f2f5d8017bfcee4d2f in 34ms, sequenceid=14, compaction requested=false 2023-07-15 11:55:55,386 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info/6c9410a40e3240ccadef6c95ab632e52, entries=24, sequenceid=31, filesize=7.4 K 2023-07-15 11:55:55,391 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 39ms, sequenceid=31, compaction requested=false 2023-07-15 11:55:55,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-07-15 11:55:55,406 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:55,406 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ed49ae2049e592f2f5d8017bfcee4d2f: 2023-07-15 11:55:55,406 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:55:55,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8bb3eb49d5c769961a9b0f7da157f1d1, disabling compactions & flushes 2023-07-15 11:55:55,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:55,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:55,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. after waiting 0 ms 2023-07-15 11:55:55,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:55,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8bb3eb49d5c769961a9b0f7da157f1d1 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-07-15 11:55:55,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:55,410 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-15 11:55:55,411 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-15 11:55:55,412 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-15 11:55:55,412 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-15 11:55:55,412 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-15 11:55:55,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=16 (bloomFilter=true), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/.tmp/cf1/5cfa9235cb204ad1917d694f48598c17 2023-07-15 11:55:55,428 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5cfa9235cb204ad1917d694f48598c17 2023-07-15 11:55:55,429 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/.tmp/cf1/5cfa9235cb204ad1917d694f48598c17 as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1/5cfa9235cb204ad1917d694f48598c17 2023-07-15 11:55:55,432 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:55,436 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5cfa9235cb204ad1917d694f48598c17 2023-07-15 11:55:55,436 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1/5cfa9235cb204ad1917d694f48598c17, entries=100, sequenceid=16, filesize=7.9 K 2023-07-15 11:55:55,437 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for 8bb3eb49d5c769961a9b0f7da157f1d1 in 30ms, sequenceid=16, compaction requested=false 2023-07-15 11:55:55,447 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:55,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/recovered.edits/19.seqid, newMaxSeqId=19, maxSeqId=12 2023-07-15 11:55:55,450 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:55,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8bb3eb49d5c769961a9b0f7da157f1d1: 2023-07-15 11:55:55,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:55:55,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing faf8bf9c0e6d77a0bdf0f3a70ab43734, disabling compactions & flushes 2023-07-15 11:55:55,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:55,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:55,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. after waiting 0 ms 2023-07-15 11:55:55,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:55,456 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-15 11:55:55,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:55,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for faf8bf9c0e6d77a0bdf0f3a70ab43734: 2023-07-15 11:55:55,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:55:55,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:55,536 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:55,552 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33403,1689422132751; all regions closed. 2023-07-15 11:55:55,553 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:55,561 DEBUG [RS:0;jenkins-hbase4:33403] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:55,561 INFO [RS:0;jenkins-hbase4:33403] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C33403%2C1689422132751.meta:.meta(num 1689422133307) 2023-07-15 11:55:55,561 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:55,569 DEBUG [RS:0;jenkins-hbase4:33403] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:55,569 INFO [RS:0;jenkins-hbase4:33403] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C33403%2C1689422132751.jenkins-hbase4.apache.org%2C33403%2C1689422132751.regiongroup-1:(num 1689422137701) 2023-07-15 11:55:55,569 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:55,576 DEBUG [RS:0;jenkins-hbase4:33403] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:55:55,576 INFO [RS:0;jenkins-hbase4:33403] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C33403%2C1689422132751.jenkins-hbase4.apache.org%2C33403%2C1689422132751.regiongroup-0:(num 1689422133155) 2023-07-15 11:55:55,576 DEBUG [RS:0;jenkins-hbase4:33403] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:55,576 INFO [RS:0;jenkins-hbase4:33403] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:55,576 INFO [RS:0;jenkins-hbase4:33403] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-15 11:55:55,576 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:55,576 DEBUG [RS:0;jenkins-hbase4:33403] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x65599ce2 to 127.0.0.1:57666 2023-07-15 11:55:55,577 DEBUG [RS:0;jenkins-hbase4:33403] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:55,577 INFO [RS:0;jenkins-hbase4:33403] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33403 2023-07-15 11:55:55,582 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33403,1689422132751 2023-07-15 11:55:55,582 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520026, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:55,582 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:55,582 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-15 11:55:55,582 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:55,583 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33403,1689422132751] 2023-07-15 11:55:55,583 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33403,1689422132751; numProcessing=1 2023-07-15 11:55:55,584 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,33403,1689422132751 already deleted, retry=false 2023-07-15 11:55:55,584 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33403,1689422132751 expired; onlineServers=0 2023-07-15 11:55:55,584 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,35607,1689422132725' ***** 2023-07-15 11:55:55,584 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-15 11:55:55,585 DEBUG [M:0;jenkins-hbase4:35607] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78c4c536, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:55,585 INFO [M:0;jenkins-hbase4:35607] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:55,585 INFO [M:0;jenkins-hbase4:35607] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,35607,1689422132725; all regions closed. 2023-07-15 11:55:55,585 DEBUG [M:0;jenkins-hbase4:35607] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:55,585 DEBUG [M:0;jenkins-hbase4:35607] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-15 11:55:55,585 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-15 11:55:55,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422132968] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422132968,5,FailOnTimeoutGroup] 2023-07-15 11:55:55,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422132968] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422132968,5,FailOnTimeoutGroup] 2023-07-15 11:55:55,585 DEBUG [M:0;jenkins-hbase4:35607] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-15 11:55:55,587 INFO [M:0;jenkins-hbase4:35607] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-15 11:55:55,587 INFO [M:0;jenkins-hbase4:35607] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-15 11:55:55,587 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-15 11:55:55,587 INFO [M:0;jenkins-hbase4:35607] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-15 11:55:55,587 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:55,587 DEBUG [M:0;jenkins-hbase4:35607] master.HMaster(1512): Stopping service threads 2023-07-15 11:55:55,587 INFO [M:0;jenkins-hbase4:35607] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-15 11:55:55,588 ERROR [M:0;jenkins-hbase4:35607] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-15 11:55:55,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:55,588 INFO [M:0;jenkins-hbase4:35607] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-15 11:55:55,588 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-15 11:55:55,588 DEBUG [M:0;jenkins-hbase4:35607] zookeeper.ZKUtil(398): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-15 11:55:55,588 WARN [M:0;jenkins-hbase4:35607] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-15 11:55:55,588 INFO [M:0;jenkins-hbase4:35607] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-15 11:55:55,589 INFO [M:0;jenkins-hbase4:35607] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-15 11:55:55,590 DEBUG [M:0;jenkins-hbase4:35607] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-15 11:55:55,590 INFO [M:0;jenkins-hbase4:35607] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:55,590 DEBUG [M:0;jenkins-hbase4:35607] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:55,590 DEBUG [M:0;jenkins-hbase4:35607] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-15 11:55:55,590 DEBUG [M:0;jenkins-hbase4:35607] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:55,590 INFO [M:0;jenkins-hbase4:35607] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=35.97 KB heapSize=44.80 KB 2023-07-15 11:55:55,602 INFO [M:0;jenkins-hbase4:35607] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=35.97 KB at sequenceid=219 (bloomFilter=true), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/98ca59b7f2c24065b5f77ddbc02f622f 2023-07-15 11:55:55,607 DEBUG [M:0;jenkins-hbase4:35607] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/98ca59b7f2c24065b5f77ddbc02f622f as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/98ca59b7f2c24065b5f77ddbc02f622f 2023-07-15 11:55:55,612 INFO [M:0;jenkins-hbase4:35607] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/98ca59b7f2c24065b5f77ddbc02f622f, entries=12, sequenceid=219, filesize=6.4 K 2023-07-15 11:55:55,613 INFO [M:0;jenkins-hbase4:35607] regionserver.HRegion(2948): Finished flush of dataSize ~35.97 KB/36838, heapSize ~44.78 KB/45856, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=219, compaction requested=false 2023-07-15 11:55:55,615 INFO [M:0;jenkins-hbase4:35607] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:55,615 DEBUG [M:0;jenkins-hbase4:35607] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:55,615 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,35607,1689422132725 2023-07-15 11:55:55,623 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:55,623 INFO [M:0;jenkins-hbase4:35607] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-15 11:55:55,624 INFO [M:0;jenkins-hbase4:35607] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35607 2023-07-15 11:55:55,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:55,628 DEBUG [M:0;jenkins-hbase4:35607] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,35607,1689422132725 already deleted, retry=false 2023-07-15 11:55:55,640 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:55,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:55,744 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:55,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:55,844 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:55,844 INFO [M:0;jenkins-hbase4:35607] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,35607,1689422132725; zookeeper connection closed. 2023-07-15 11:55:55,845 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:35607-0x10168d63052001c, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:55,850 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:55,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-15 11:55:55,945 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:55,945 INFO [RS:0;jenkins-hbase4:33403] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33403,1689422132751; zookeeper connection closed. 2023-07-15 11:55:55,945 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:33403-0x10168d63052001d, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:55,946 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6cb67ec1] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6cb67ec1 2023-07-15 11:55:55,946 INFO [Listener at localhost/46025] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-15 11:55:55,948 DEBUG [hconnection-0x6ff5a416-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:55:55,950 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42400, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:55:55,957 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:56,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 9797 2023-07-15 11:55:56,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 85 WAL entries eligible for replication 2023-07-15 11:55:56,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 9797 2023-07-15 11:55:56,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/eee02c0767aa2be4d83f125ea78097e3/112=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/113=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/114=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/115=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/116=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/117=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/118=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/119=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/120=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/121=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/122=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/123=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/124=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/125=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/126=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/127=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/128=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/129=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/130=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/131=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/132=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/133=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/134=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/135=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/136=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/137=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/138=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/139=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/140=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/141=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/142=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/143=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/144=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/145=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/146=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/147=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/148=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/149=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/150=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/151=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/152=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/153=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/154=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/155=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/156=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/157=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/158=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/159=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/160=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/161=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/162=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/163=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/164=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/165=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/166=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/167=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/168=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/169=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/170=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/171=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/172=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/173=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/174=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/175=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/176=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/177=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/178=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/179=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/180=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/181=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/182=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/183=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/184=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/185=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/186=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/187=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/188=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/189=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/190=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/191=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/192=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/193=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/194=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/195=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/196=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1.1689422154536, lastWalPosition=9797, nbRowKeys=85, nbHFiles=0, heapSize=17680, lastSeqIds={}, endOfFile=false,usedBufferSize=17680] 2023-07-15 11:55:56,056 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-15 11:55:56,062 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:56,157 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-15 11:55:56,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 16143 2023-07-15 11:55:56,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 59 WAL entries eligible for replication 2023-07-15 11:55:56,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 16143 2023-07-15 11:55:56,188 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:56,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 26183 2023-07-15 11:55:56,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 94 WAL entries eligible for replication 2023-07-15 11:55:56,293 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:56,343 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x49f409d4 to 127.0.0.1:57666 2023-07-15 11:55:56,343 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:56,343 DEBUG [Listener at localhost/46025] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-15 11:55:56,344 DEBUG [Listener at localhost/46025] util.JVMClusterUtil(257): Found active master hash=1527247909, stopped=false 2023-07-15 11:55:56,344 INFO [Listener at localhost/46025] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:56,345 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-15 11:55:56,345 INFO [Listener at localhost/46025] procedure2.ProcedureExecutor(629): Stopping 2023-07-15 11:55:56,345 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-15 11:55:56,346 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:56,348 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-15 11:55:56,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-15 11:55:56,349 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x55854855 to 127.0.0.1:57666 2023-07-15 11:55:56,349 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:56,349 INFO [Listener at localhost/46025] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46571,1689422149499' ***** 2023-07-15 11:55:56,349 INFO [Listener at localhost/46025] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-15 11:55:56,350 INFO [RS:0;jenkins-hbase4:46571] regionserver.HeapMemoryManager(220): Stopping 2023-07-15 11:55:56,350 INFO [RS:0;jenkins-hbase4:46571] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-15 11:55:56,350 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-15 11:55:56,350 INFO [RS:0;jenkins-hbase4:46571] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-15 11:55:56,351 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(3305): Received CLOSE for 89b547022cb2e9610cb6246270f3cbaf 2023-07-15 11:55:56,351 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(3305): Received CLOSE for f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:56,351 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(3305): Received CLOSE for eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:56,351 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:56,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 89b547022cb2e9610cb6246270f3cbaf, disabling compactions & flushes 2023-07-15 11:55:56,352 DEBUG [RS:0;jenkins-hbase4:46571] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0d1153b8 to 127.0.0.1:57666 2023-07-15 11:55:56,352 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:56,352 DEBUG [RS:0;jenkins-hbase4:46571] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:56,352 INFO [RS:0;jenkins-hbase4:46571] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-15 11:55:56,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:56,352 INFO [RS:0;jenkins-hbase4:46571] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-15 11:55:56,353 INFO [RS:0;jenkins-hbase4:46571] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-15 11:55:56,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. after waiting 0 ms 2023-07-15 11:55:56,353 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-15 11:55:56,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:56,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 89b547022cb2e9610cb6246270f3cbaf 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-07-15 11:55:56,353 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-15 11:55:56,353 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1478): Online Regions={89b547022cb2e9610cb6246270f3cbaf=t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf., f1407b2bc354da8a7c08c95abb9ab378=hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378., 1588230740=hbase:meta,,1.1588230740, eee02c0767aa2be4d83f125ea78097e3=t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3.} 2023-07-15 11:55:56,354 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1504): Waiting on 1588230740, 89b547022cb2e9610cb6246270f3cbaf, eee02c0767aa2be4d83f125ea78097e3, f1407b2bc354da8a7c08c95abb9ab378 2023-07-15 11:55:56,354 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-15 11:55:56,354 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-15 11:55:56,354 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-15 11:55:56,354 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-15 11:55:56,354 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-15 11:55:56,355 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-07-15 11:55:56,357 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-15 11:55:56,371 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/.tmp/info/37b2b45961e14df78bae813d9d56f018 2023-07-15 11:55:56,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/.tmp/cf1/782f671f29454f688ca232cd68276780 2023-07-15 11:55:56,389 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:56,395 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/.tmp/rep_barrier/bf98d864e244497fadd52f4755df3199 2023-07-15 11:55:56,395 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/.tmp/norep/ab9cb711642945ceabeee385778c72ef 2023-07-15 11:55:56,399 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-15 11:55:56,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/.tmp/cf1/782f671f29454f688ca232cd68276780 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/cf1/782f671f29454f688ca232cd68276780 2023-07-15 11:55:56,408 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/.tmp/info/37b2b45961e14df78bae813d9d56f018 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/info/37b2b45961e14df78bae813d9d56f018 2023-07-15 11:55:56,415 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/info/37b2b45961e14df78bae813d9d56f018, entries=24, sequenceid=31, filesize=7.4 K 2023-07-15 11:55:56,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/cf1/782f671f29454f688ca232cd68276780, entries=200, sequenceid=413, filesize=12.5 K 2023-07-15 11:55:56,416 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/.tmp/rep_barrier/bf98d864e244497fadd52f4755df3199 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/rep_barrier/bf98d864e244497fadd52f4755df3199 2023-07-15 11:55:56,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/.tmp/norep/ab9cb711642945ceabeee385778c72ef as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/norep/ab9cb711642945ceabeee385778c72ef 2023-07-15 11:55:56,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/norep/ab9cb711642945ceabeee385778c72ef, entries=1, sequenceid=413, filesize=4.8 K 2023-07-15 11:55:56,427 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/rep_barrier/bf98d864e244497fadd52f4755df3199, entries=2, sequenceid=31, filesize=4.9 K 2023-07-15 11:55:56,428 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for 89b547022cb2e9610cb6246270f3cbaf in 75ms, sequenceid=413, compaction requested=false 2023-07-15 11:55:56,429 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 74ms, sequenceid=31, compaction requested=false 2023-07-15 11:55:56,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t2_syncup/89b547022cb2e9610cb6246270f3cbaf/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-07-15 11:55:56,463 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-15 11:55:56,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:56,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 89b547022cb2e9610cb6246270f3cbaf: 2023-07-15 11:55:56,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689422126158.89b547022cb2e9610cb6246270f3cbaf. 2023-07-15 11:55:56,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f1407b2bc354da8a7c08c95abb9ab378, disabling compactions & flushes 2023-07-15 11:55:56,465 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:56,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:56,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. after waiting 0 ms 2023-07-15 11:55:56,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:56,471 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-15 11:55:56,471 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-15 11:55:56,471 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-15 11:55:56,471 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-15 11:55:56,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/hbase/namespace/f1407b2bc354da8a7c08c95abb9ab378/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-15 11:55:56,476 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:56,476 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f1407b2bc354da8a7c08c95abb9ab378: 2023-07-15 11:55:56,476 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689422121725.f1407b2bc354da8a7c08c95abb9ab378. 2023-07-15 11:55:56,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing eee02c0767aa2be4d83f125ea78097e3, disabling compactions & flushes 2023-07-15 11:55:56,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:56,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:56,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. after waiting 0 ms 2023-07-15 11:55:56,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:56,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing eee02c0767aa2be4d83f125ea78097e3 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-07-15 11:55:56,540 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-15 11:55:56,540 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-15 11:55:56,540 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/.tmp/cf1/86dce89592444abdbb731369a830c12d 2023-07-15 11:55:56,540 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-15 11:55:56,540 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0.1689422149924, lastWalPosition=728, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-15 11:55:56,554 DEBUG [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1504): Waiting on eee02c0767aa2be4d83f125ea78097e3 2023-07-15 11:55:56,571 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/.tmp/norep/543c270515a14968a504f696c84443fd 2023-07-15 11:55:56,580 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/.tmp/cf1/86dce89592444abdbb731369a830c12d as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/cf1/86dce89592444abdbb731369a830c12d 2023-07-15 11:55:56,588 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/cf1/86dce89592444abdbb731369a830c12d, entries=100, sequenceid=213, filesize=8.4 K 2023-07-15 11:55:56,589 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/.tmp/norep/543c270515a14968a504f696c84443fd as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/norep/543c270515a14968a504f696c84443fd 2023-07-15 11:55:56,593 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/norep/543c270515a14968a504f696c84443fd, entries=1, sequenceid=213, filesize=4.8 K 2023-07-15 11:55:56,594 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for eee02c0767aa2be4d83f125ea78097e3 in 116ms, sequenceid=213, compaction requested=false 2023-07-15 11:55:56,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/data/default/t1_syncup/eee02c0767aa2be4d83f125ea78097e3/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-07-15 11:55:56,603 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:56,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for eee02c0767aa2be4d83f125ea78097e3: 2023-07-15 11:55:56,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689422125521.eee02c0767aa2be4d83f125ea78097e3. 2023-07-15 11:55:56,648 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-15 11:55:56,658 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-15 11:55:56,752 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-15 11:55:56,754 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46571,1689422149499; all regions closed. 2023-07-15 11:55:56,755 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:56,761 DEBUG [RS:0;jenkins-hbase4:46571] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs 2023-07-15 11:55:56,761 INFO [RS:0;jenkins-hbase4:46571] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C46571%2C1689422149499.meta:.meta(num 1689422150002) 2023-07-15 11:55:56,761 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:56,766 DEBUG [RS:0;jenkins-hbase4:46571] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs 2023-07-15 11:55:56,766 INFO [RS:0;jenkins-hbase4:46571] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1:(num 1689422154536) 2023-07-15 11:55:56,766 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:56,771 DEBUG [RS:0;jenkins-hbase4:46571] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs 2023-07-15 11:55:56,771 INFO [RS:0;jenkins-hbase4:46571] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0:(num 1689422149924) 2023-07-15 11:55:56,771 DEBUG [RS:0;jenkins-hbase4:46571] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:56,771 INFO [RS:0;jenkins-hbase4:46571] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:55:56,771 INFO [RS:0;jenkins-hbase4:46571] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-15 11:55:56,771 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:56,772 INFO [RS:0;jenkins-hbase4:46571] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-15 11:55:56,854 INFO [RS:0;jenkins-hbase4:46571.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/WALs/jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0.1689422149924 was moved to hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0.1689422149924 2023-07-15 11:55:56,874 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d63052002d, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:56,874 DEBUG [RS:0;jenkins-hbase4:46571] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x34901999 to 127.0.0.1:57666 2023-07-15 11:55:56,874 DEBUG [RS:0;jenkins-hbase4:46571.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d63052002d, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:56,874 DEBUG [RS:0;jenkins-hbase4:46571] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:56,975 INFO [RS:0;jenkins-hbase4:46571] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase4:46571.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1 terminated 2023-07-15 11:55:56,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(243): Waiting for peers [Source for peer 1]: Interrupted while sleeping between retries 2023-07-15 11:55:56,977 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-07-15 11:55:56,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(243): No sinks available at peer [Source for peer 1]: Interrupted while sleeping between retries 2023-07-15 11:55:56,977 INFO [RS:0;jenkins-hbase4:46571] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1 terminated 2023-07-15 11:55:56,978 INFO [RS:0;jenkins-hbase4:46571] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46571 2023-07-15 11:55:56,981 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:55:56,981 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-15 11:55:56,981 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-15 11:55:56,982 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46571,1689422149499] 2023-07-15 11:55:56,982 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46571,1689422149499; numProcessing=1 2023-07-15 11:55:56,983 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,46571,1689422149499 already deleted, retry=false 2023-07-15 11:55:56,983 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46571,1689422149499 expired; onlineServers=0 2023-07-15 11:55:56,983 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42779,1689422149474' ***** 2023-07-15 11:55:56,983 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-15 11:55:56,984 DEBUG [M:0;jenkins-hbase4:42779] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5f386b9d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:56,984 INFO [M:0;jenkins-hbase4:42779] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:56,984 INFO [M:0;jenkins-hbase4:42779] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42779,1689422149474; all regions closed. 2023-07-15 11:55:56,984 DEBUG [M:0;jenkins-hbase4:42779] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:55:56,984 DEBUG [M:0;jenkins-hbase4:42779] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-15 11:55:56,984 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-15 11:55:56,984 DEBUG [M:0;jenkins-hbase4:42779] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-15 11:55:56,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422149671] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422149671,5,FailOnTimeoutGroup] 2023-07-15 11:55:56,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422149668] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422149668,5,FailOnTimeoutGroup] 2023-07-15 11:55:56,985 INFO [M:0;jenkins-hbase4:42779] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-15 11:55:56,986 INFO [M:0;jenkins-hbase4:42779] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-15 11:55:56,986 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-15 11:55:56,986 INFO [M:0;jenkins-hbase4:42779] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-15 11:55:56,986 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-15 11:55:56,986 DEBUG [M:0;jenkins-hbase4:42779] master.HMaster(1512): Stopping service threads 2023-07-15 11:55:56,986 INFO [M:0;jenkins-hbase4:42779] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-15 11:55:56,987 ERROR [M:0;jenkins-hbase4:42779] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-15 11:55:56,987 INFO [M:0;jenkins-hbase4:42779] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-15 11:55:56,987 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-15 11:55:56,987 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-15 11:55:56,987 DEBUG [M:0;jenkins-hbase4:42779] zookeeper.ZKUtil(398): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-15 11:55:56,988 WARN [M:0;jenkins-hbase4:42779] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-15 11:55:56,988 INFO [M:0;jenkins-hbase4:42779] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-15 11:55:56,988 INFO [M:0;jenkins-hbase4:42779] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-15 11:55:56,990 DEBUG [M:0;jenkins-hbase4:42779] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-15 11:55:56,990 INFO [M:0;jenkins-hbase4:42779] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:56,990 DEBUG [M:0;jenkins-hbase4:42779] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:56,990 DEBUG [M:0;jenkins-hbase4:42779] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-15 11:55:56,990 DEBUG [M:0;jenkins-hbase4:42779] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:56,990 INFO [M:0;jenkins-hbase4:42779] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.40 KB heapSize=36.57 KB 2023-07-15 11:55:57,000 INFO [M:0;jenkins-hbase4:42779] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.40 KB at sequenceid=217 (bloomFilter=true), to=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a2687a2abfe54fdfbc2c22afd2fe9a89 2023-07-15 11:55:57,005 DEBUG [M:0;jenkins-hbase4:42779] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a2687a2abfe54fdfbc2c22afd2fe9a89 as hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a2687a2abfe54fdfbc2c22afd2fe9a89 2023-07-15 11:55:57,009 INFO [M:0;jenkins-hbase4:42779] regionserver.HStore(1080): Added hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a2687a2abfe54fdfbc2c22afd2fe9a89, entries=10, sequenceid=217, filesize=5.8 K 2023-07-15 11:55:57,010 INFO [M:0;jenkins-hbase4:42779] regionserver.HRegion(2948): Finished flush of dataSize ~29.40 KB/30105, heapSize ~36.55 KB/37432, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=217, compaction requested=false 2023-07-15 11:55:57,012 INFO [M:0;jenkins-hbase4:42779] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:55:57,012 DEBUG [M:0;jenkins-hbase4:42779] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:57,013 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/MasterData/WALs/jenkins-hbase4.apache.org,42779,1689422149474 2023-07-15 11:55:57,017 INFO [M:0;jenkins-hbase4:42779] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-15 11:55:57,017 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:55:57,018 INFO [M:0;jenkins-hbase4:42779] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42779 2023-07-15 11:55:57,024 DEBUG [M:0;jenkins-hbase4:42779] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,42779,1689422149474 already deleted, retry=false 2023-07-15 11:55:57,082 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:57,082 INFO [RS:0;jenkins-hbase4:46571] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46571,1689422149499; zookeeper connection closed. 2023-07-15 11:55:57,083 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:46571-0x10168d630520029, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:57,084 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6621abc3] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6621abc3 2023-07-15 11:55:57,084 INFO [Listener at localhost/46025] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-15 11:55:57,183 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:57,183 INFO [M:0;jenkins-hbase4:42779] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42779,1689422149474; zookeeper connection closed. 2023-07-15 11:55:57,183 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:42779-0x10168d630520028, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:55:57,187 INFO [Listener at localhost/46025] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-15 11:55:57,187 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:57,187 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:57,187 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:57,187 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:57,187 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:57,187 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:57,189 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39715 2023-07-15 11:55:57,189 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:57,191 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:57,192 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39715 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:57,194 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:397150x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:57,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:39715-0x10168d630520030 connected 2023-07-15 11:55:57,200 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:57,200 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:57,201 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-15 11:55:57,204 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39715 2023-07-15 11:55:57,204 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39715 2023-07-15 11:55:57,205 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39715 2023-07-15 11:55:57,205 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39715 2023-07-15 11:55:57,205 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39715 2023-07-15 11:55:57,205 INFO [Listener at localhost/46025] master.HMaster(444): hbase.rootdir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346, hbase.cluster.distributed=false 2023-07-15 11:55:57,206 DEBUG [pool-660-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-15 11:55:57,219 INFO [Listener at localhost/46025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-15 11:55:57,219 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:57,219 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:57,220 INFO [Listener at localhost/46025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-15 11:55:57,220 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-15 11:55:57,220 INFO [Listener at localhost/46025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-15 11:55:57,220 INFO [Listener at localhost/46025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-15 11:55:57,223 INFO [Listener at localhost/46025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41201 2023-07-15 11:55:57,224 INFO [Listener at localhost/46025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-15 11:55:57,225 DEBUG [Listener at localhost/46025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-15 11:55:57,226 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:57,227 INFO [Listener at localhost/46025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:57,228 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41201 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:55:57,232 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:412010x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:55:57,233 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:412010x0, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:55:57,234 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:412010x0, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:55:57,234 DEBUG [Listener at localhost/46025] zookeeper.ZKUtil(164): regionserver:412010x0, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-15 11:55:57,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41201-0x10168d630520031 connected 2023-07-15 11:55:57,239 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41201 2023-07-15 11:55:57,239 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41201 2023-07-15 11:55:57,246 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41201 2023-07-15 11:55:57,246 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41201 2023-07-15 11:55:57,247 DEBUG [Listener at localhost/46025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41201 2023-07-15 11:55:57,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:55:57,260 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-15 11:55:57,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:55:57,267 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-15 11:55:57,267 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:41201-0x10168d630520031, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-15 11:55:57,267 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:57,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-15 11:55:57,271 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,39715,1689422157186 from backup master directory 2023-07-15 11:55:57,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-15 11:55:57,273 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:55:57,273 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:57,273 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-15 11:55:57,273 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:55:57,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-15 11:55:57,318 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4eb9c718 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:57,328 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@248a8e76, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:57,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-15 11:55:57,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-15 11:55:57,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:57,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:57,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,35607,1689422132725 to hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,35607,1689422132725-dead as it is dead 2023-07-15 11:55:57,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,35607,1689422132725-dead/jenkins-hbase4.apache.org%2C35607%2C1689422132725.jenkins-hbase4.apache.org%2C35607%2C1689422132725.regiongroup-0.1689422132853 2023-07-15 11:55:57,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,35607,1689422132725-dead/jenkins-hbase4.apache.org%2C35607%2C1689422132725.jenkins-hbase4.apache.org%2C35607%2C1689422132725.regiongroup-0.1689422132853 after 0ms 2023-07-15 11:55:57,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,35607,1689422132725-dead/jenkins-hbase4.apache.org%2C35607%2C1689422132725.jenkins-hbase4.apache.org%2C35607%2C1689422132725.regiongroup-0.1689422132853 to hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C35607%2C1689422132725.jenkins-hbase4.apache.org%2C35607%2C1689422132725.regiongroup-0.1689422132853 2023-07-15 11:55:57,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,35607,1689422132725-dead 2023-07-15 11:55:57,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:55:57,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:57,340 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39715%2C1689422157186.jenkins-hbase4.apache.org%2C39715%2C1689422157186.regiongroup-0, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,39715,1689422157186, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/oldWALs, maxLogs=10 2023-07-15 11:55:57,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,39715,1689422157186/jenkins-hbase4.apache.org%2C39715%2C1689422157186.jenkins-hbase4.apache.org%2C39715%2C1689422157186.regiongroup-0.1689422157341 2023-07-15 11:55:57,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-274df9f8-c308-48fc-8592-c230a3934cdd,DISK]] 2023-07-15 11:55:57,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:57,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:57,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:57,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:57,349 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:57,350 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-15 11:55:57,350 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-15 11:55:57,356 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/87fe3d1a2e1b45d28817fd4987a3cba4 2023-07-15 11:55:57,360 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/98ca59b7f2c24065b5f77ddbc02f622f 2023-07-15 11:55:57,360 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:57,361 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-15 11:55:57,361 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C35607%2C1689422132725.jenkins-hbase4.apache.org%2C35607%2C1689422132725.regiongroup-0.1689422132853 2023-07-15 11:55:57,365 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 117, firstSequenceIdInLog=120, maxSequenceIdInLog=221, path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C35607%2C1689422132725.jenkins-hbase4.apache.org%2C35607%2C1689422132725.regiongroup-0.1689422132853 2023-07-15 11:55:57,365 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C35607%2C1689422132725.jenkins-hbase4.apache.org%2C35607%2C1689422132725.regiongroup-0.1689422132853 2023-07-15 11:55:57,368 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-15 11:55:57,370 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/221.seqid, newMaxSeqId=221, maxSeqId=118 2023-07-15 11:55:57,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=222; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72295169, jitterRate=0.07728196680545807}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-15 11:55:57,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:55:57,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-15 11:55:57,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-15 11:55:57,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-15 11:55:57,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-15 11:55:57,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-15 11:55:57,379 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-15 11:55:57,380 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-15 11:55:57,380 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-15 11:55:57,380 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-15 11:55:57,381 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-15 11:55:57,381 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-15 11:55:57,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46827,1689422123904, splitWal=true, meta=true 2023-07-15 11:55:57,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,39401,1689422123877, splitWal=true, meta=false 2023-07-15 11:55:57,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43589,1689422123957, splitWal=true, meta=false 2023-07-15 11:55:57,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45535,1689422123929, splitWal=true, meta=false 2023-07-15 11:55:57,383 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-15 11:55:57,383 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-15 11:55:57,383 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-15 11:55:57,384 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33403,1689422132751, table=hbase:meta, region=1588230740 2023-07-15 11:55:57,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-07-15 11:55:57,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,33403,1689422132751 already deleted, retry=false 2023-07-15 11:55:57,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,33403,1689422132751 on jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:55:57,389 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,33403,1689422132751, splitWal=true, meta=true 2023-07-15 11:55:57,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase4.apache.org,33403,1689422132751 (carryingMeta=true) jenkins-hbase4.apache.org,33403,1689422132751/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@74a14ed9[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-15 11:55:57,405 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-15 11:55:57,405 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-15 11:55:57,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-15 11:55:57,406 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-15 11:55:57,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-15 11:55:57,407 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-15 11:55:57,407 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-15 11:55:57,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-15 11:55:57,413 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-15 11:55:57,413 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:41201-0x10168d630520031, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-15 11:55:57,413 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:55:57,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39715,1689422157186, sessionid=0x10168d630520030, setting cluster-up flag (Was=false) 2023-07-15 11:55:57,418 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-15 11:55:57,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:55:57,421 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-15 11:55:57,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:55:57,422 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/.hbase-snapshot/.tmp 2023-07-15 11:55:57,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:57,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:57,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:57,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-15 11:55:57,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-15 11:55:57,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:57,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689422187436 2023-07-15 11:55:57,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-15 11:55:57,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-15 11:55:57,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-15 11:55:57,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-15 11:55:57,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-15 11:55:57,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-15 11:55:57,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:57,438 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33403,1689422132751; numProcessing=1 2023-07-15 11:55:57,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-15 11:55:57,438 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33403,1689422132751, splitWal=true, meta=true 2023-07-15 11:55:57,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-15 11:55:57,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-15 11:55:57,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-15 11:55:57,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-15 11:55:57,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422157440,5,FailOnTimeoutGroup] 2023-07-15 11:55:57,441 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422157440,5,FailOnTimeoutGroup] 2023-07-15 11:55:57,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:57,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-15 11:55:57,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:57,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:57,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689422157441, completionTime=-1 2023-07-15 11:55:57,441 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-15 11:55:57,441 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33403,1689422132751, splitWal=true, meta=true, isMeta: true 2023-07-15 11:55:57,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-15 11:55:57,443 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751-splitting 2023-07-15 11:55:57,443 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751-splitting dir is empty, no logs to split. 2023-07-15 11:55:57,443 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,33403,1689422132751 WAL count=0, meta=true 2023-07-15 11:55:57,445 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751-splitting dir is empty, no logs to split. 2023-07-15 11:55:57,445 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,33403,1689422132751 WAL count=0, meta=true 2023-07-15 11:55:57,445 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,33403,1689422132751 WAL splitting is done? wals=0, meta=true 2023-07-15 11:55:57,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-15 11:55:57,448 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-15 11:55:57,449 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:55:57,455 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(951): ClusterId : e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:55:57,455 DEBUG [RS:0;jenkins-hbase4:41201] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-15 11:55:57,457 DEBUG [RS:0;jenkins-hbase4:41201] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-15 11:55:57,457 DEBUG [RS:0;jenkins-hbase4:41201] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-15 11:55:57,461 DEBUG [RS:0;jenkins-hbase4:41201] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-15 11:55:57,463 DEBUG [RS:0;jenkins-hbase4:41201] zookeeper.ReadOnlyZKClient(139): Connect 0x547904a8 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:55:57,469 DEBUG [RS:0;jenkins-hbase4:41201] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5abee057, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:55:57,469 DEBUG [RS:0;jenkins-hbase4:41201] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e816fd8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:55:57,477 DEBUG [RS:0;jenkins-hbase4:41201] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:41201 2023-07-15 11:55:57,478 INFO [RS:0;jenkins-hbase4:41201] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-15 11:55:57,478 INFO [RS:0;jenkins-hbase4:41201] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-15 11:55:57,478 DEBUG [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1022): About to register with Master. 2023-07-15 11:55:57,478 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39715,1689422157186 with isa=jenkins-hbase4.apache.org/172.31.14.131:41201, startcode=1689422157218 2023-07-15 11:55:57,478 DEBUG [RS:0;jenkins-hbase4:41201] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-15 11:55:57,482 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51911, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-15 11:55:57,483 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39715] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:55:57,483 DEBUG [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346 2023-07-15 11:55:57,483 DEBUG [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46809 2023-07-15 11:55:57,483 DEBUG [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-15 11:55:57,486 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520026, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:57,486 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:55:57,486 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-15 11:55:57,487 DEBUG [RS:0;jenkins-hbase4:41201] zookeeper.ZKUtil(162): regionserver:41201-0x10168d630520031, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:55:57,487 WARN [RS:0;jenkins-hbase4:41201] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-15 11:55:57,487 INFO [RS:0;jenkins-hbase4:41201] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:57,487 INFO [RS:0;jenkins-hbase4:41201] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:57,487 DEBUG [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:55:57,488 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41201,1689422157218] 2023-07-15 11:55:57,491 DEBUG [RS:0;jenkins-hbase4:41201] zookeeper.ZKUtil(162): regionserver:41201-0x10168d630520031, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:55:57,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=51ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-15 11:55:57,492 DEBUG [RS:0;jenkins-hbase4:41201] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:55:57,492 INFO [RS:0;jenkins-hbase4:41201] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-15 11:55:57,493 INFO [RS:0;jenkins-hbase4:41201] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-15 11:55:57,497 INFO [RS:0;jenkins-hbase4:41201] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-15 11:55:57,497 INFO [RS:0;jenkins-hbase4:41201] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:57,498 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-15 11:55:57,500 INFO [RS:0;jenkins-hbase4:41201] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:57,500 DEBUG [RS:0;jenkins-hbase4:41201] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,500 DEBUG [RS:0;jenkins-hbase4:41201] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,500 DEBUG [RS:0;jenkins-hbase4:41201] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,500 DEBUG [RS:0;jenkins-hbase4:41201] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,500 DEBUG [RS:0;jenkins-hbase4:41201] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,500 DEBUG [RS:0;jenkins-hbase4:41201] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-15 11:55:57,500 DEBUG [RS:0;jenkins-hbase4:41201] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,500 DEBUG [RS:0;jenkins-hbase4:41201] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,500 DEBUG [RS:0;jenkins-hbase4:41201] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,500 DEBUG [RS:0;jenkins-hbase4:41201] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-15 11:55:57,502 INFO [RS:0;jenkins-hbase4:41201] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:57,502 INFO [RS:0;jenkins-hbase4:41201] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:57,502 INFO [RS:0;jenkins-hbase4:41201] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:57,513 INFO [RS:0;jenkins-hbase4:41201] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-15 11:55:57,513 INFO [RS:0;jenkins-hbase4:41201] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41201,1689422157218-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:55:57,524 INFO [RS:0;jenkins-hbase4:41201] regionserver.Replication(203): jenkins-hbase4.apache.org,41201,1689422157218 started 2023-07-15 11:55:57,524 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41201,1689422157218, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41201, sessionid=0x10168d630520031 2023-07-15 11:55:57,524 DEBUG [RS:0;jenkins-hbase4:41201] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-15 11:55:57,524 DEBUG [RS:0;jenkins-hbase4:41201] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:55:57,524 DEBUG [RS:0;jenkins-hbase4:41201] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41201,1689422157218' 2023-07-15 11:55:57,524 DEBUG [RS:0;jenkins-hbase4:41201] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-15 11:55:57,524 DEBUG [RS:0;jenkins-hbase4:41201] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-15 11:55:57,524 DEBUG [RS:0;jenkins-hbase4:41201] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-15 11:55:57,525 DEBUG [RS:0;jenkins-hbase4:41201] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-15 11:55:57,525 DEBUG [RS:0;jenkins-hbase4:41201] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:55:57,525 DEBUG [RS:0;jenkins-hbase4:41201] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41201,1689422157218' 2023-07-15 11:55:57,525 DEBUG [RS:0;jenkins-hbase4:41201] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-15 11:55:57,525 DEBUG [RS:0;jenkins-hbase4:41201] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-15 11:55:57,525 DEBUG [RS:0;jenkins-hbase4:41201] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-15 11:55:57,525 INFO [RS:0;jenkins-hbase4:41201] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-15 11:55:57,525 INFO [RS:0;jenkins-hbase4:41201] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-15 11:55:57,600 DEBUG [jenkins-hbase4:39715] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-15 11:55:57,601 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41201,1689422157218, state=OPENING 2023-07-15 11:55:57,602 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-15 11:55:57,602 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-15 11:55:57,602 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41201,1689422157218}] 2023-07-15 11:55:57,625 INFO [RS:0;jenkins-hbase4:41201] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:57,627 INFO [RS:0;jenkins-hbase4:41201] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41201%2C1689422157218.jenkins-hbase4.apache.org%2C41201%2C1689422157218.regiongroup-0, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,41201,1689422157218, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:57,635 INFO [RS:0;jenkins-hbase4:41201] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,41201,1689422157218/jenkins-hbase4.apache.org%2C41201%2C1689422157218.jenkins-hbase4.apache.org%2C41201%2C1689422157218.regiongroup-0.1689422157628 2023-07-15 11:55:57,635 DEBUG [RS:0;jenkins-hbase4:41201] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45131,DS-caa7d3ae-4d76-4779-b495-05a972fbb935,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK]] 2023-07-15 11:55:57,758 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:55:57,759 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:55:57,761 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40188, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-15 11:55:57,764 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-15 11:55:57,765 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:55:57,765 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:55:57,765 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:55:57,766 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41201%2C1689422157218.meta, suffix=.meta, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,41201,1689422157218, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:55:57,773 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,41201,1689422157218/jenkins-hbase4.apache.org%2C41201%2C1689422157218.meta.1689422157767.meta 2023-07-15 11:55:57,773 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-274df9f8-c308-48fc-8592-c230a3934cdd,DISK]] 2023-07-15 11:55:57,773 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:55:57,773 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-15 11:55:57,773 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-15 11:55:57,773 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-15 11:55:57,773 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-15 11:55:57,773 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:55:57,773 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-15 11:55:57,773 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-15 11:55:57,775 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-15 11:55:57,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info 2023-07-15 11:55:57,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info 2023-07-15 11:55:57,777 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-15 11:55:57,785 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info/6c9410a40e3240ccadef6c95ab632e52 2023-07-15 11:55:57,789 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info/b02e059c4d7e4d748077b55ccddf7346 2023-07-15 11:55:57,789 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:57,790 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-15 11:55:57,790 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:57,791 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/rep_barrier 2023-07-15 11:55:57,791 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-15 11:55:57,792 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:57,792 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-15 11:55:57,793 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table 2023-07-15 11:55:57,793 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table 2023-07-15 11:55:57,793 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-15 11:55:57,798 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/table/298f7c6b75f14089aed20c034ec773cc 2023-07-15 11:55:57,798 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:55:57,798 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740 2023-07-15 11:55:57,799 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740 2023-07-15 11:55:57,802 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-15 11:55:57,803 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-15 11:55:57,804 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71978967, jitterRate=0.07257018983364105}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-15 11:55:57,804 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-15 11:55:57,808 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1689422157758 2023-07-15 11:55:57,812 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-15 11:55:57,813 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-15 11:55:57,814 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41201,1689422157218, state=OPEN 2023-07-15 11:55:57,815 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-15 11:55:57,816 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-15 11:55:57,819 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-15 11:55:57,819 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41201,1689422157218 in 214 msec 2023-07-15 11:55:57,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-07-15 11:55:57,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 373 msec 2023-07-15 11:55:58,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1553ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-15 11:56:00,043 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-15 11:56:00,497 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3056ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-15 11:56:01,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4508ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-15 11:56:01,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-15 11:56:01,953 DEBUG [hconnection-0x4a1034b5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:56:01,955 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50370, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:56:01,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=faf8bf9c0e6d77a0bdf0f3a70ab43734, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33403,1689422132751, regionLocation=jenkins-hbase4.apache.org,33403,1689422132751, openSeqNum=10 2023-07-15 11:56:01,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ed49ae2049e592f2f5d8017bfcee4d2f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33403,1689422132751, regionLocation=jenkins-hbase4.apache.org,33403,1689422132751, openSeqNum=11 2023-07-15 11:56:01,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=8bb3eb49d5c769961a9b0f7da157f1d1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33403,1689422132751, regionLocation=jenkins-hbase4.apache.org,33403,1689422132751, openSeqNum=13 2023-07-15 11:56:01,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-15 11:56:01,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689422221963 2023-07-15 11:56:01,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689422281963 2023-07-15 11:56:01,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 14 msec 2023-07-15 11:56:01,981 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,33403,1689422132751 had 4 regions 2023-07-15 11:56:01,981 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39715,1689422157186-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-15 11:56:01,981 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39715,1689422157186-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:56:01,981 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39715,1689422157186-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:56:01,982 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39715, period=300000, unit=MILLISECONDS is enabled. 2023-07-15 11:56:01,982 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-15 11:56:01,982 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. is NOT online; state={faf8bf9c0e6d77a0bdf0f3a70ab43734 state=OPEN, ts=1689422161963, server=jenkins-hbase4.apache.org,33403,1689422132751}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-15 11:56:01,984 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33403,1689422132751, splitWal=true, meta=true, isMeta: false 2023-07-15 11:56:01,987 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,33403,1689422132751/hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734., unknown_server=jenkins-hbase4.apache.org,33403,1689422132751/t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f., unknown_server=jenkins-hbase4.apache.org,33403,1689422132751/t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:56:01,987 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751-splitting dir is empty, no logs to split. 2023-07-15 11:56:01,987 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,33403,1689422132751 WAL count=0, meta=false 2023-07-15 11:56:01,990 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,33403,1689422132751-splitting dir is empty, no logs to split. 2023-07-15 11:56:01,990 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,33403,1689422132751 WAL count=0, meta=false 2023-07-15 11:56:01,990 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,33403,1689422132751 WAL splitting is done? wals=0, meta=false 2023-07-15 11:56:01,994 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN}] 2023-07-15 11:56:01,996 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN 2023-07-15 11:56:01,996 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN 2023-07-15 11:56:01,996 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN 2023-07-15 11:56:01,997 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:56:01,997 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:56:01,997 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-15 11:56:02,147 DEBUG [jenkins-hbase4:39715] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-15 11:56:02,151 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=8bb3eb49d5c769961a9b0f7da157f1d1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:02,151 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=ed49ae2049e592f2f5d8017bfcee4d2f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:02,151 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=faf8bf9c0e6d77a0bdf0f3a70ab43734, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:02,151 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422162151"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422162151"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422162151"}]},"ts":"1689422162151"} 2023-07-15 11:56:02,151 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422162151"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422162151"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422162151"}]},"ts":"1689422162151"} 2023-07-15 11:56:02,151 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422162151"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689422162151"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689422162151"}]},"ts":"1689422162151"} 2023-07-15 11:56:02,154 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE; OpenRegionProcedure 8bb3eb49d5c769961a9b0f7da157f1d1, server=jenkins-hbase4.apache.org,41201,1689422157218}] 2023-07-15 11:56:02,155 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure faf8bf9c0e6d77a0bdf0f3a70ab43734, server=jenkins-hbase4.apache.org,41201,1689422157218}] 2023-07-15 11:56:02,157 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=31, state=RUNNABLE; OpenRegionProcedure ed49ae2049e592f2f5d8017bfcee4d2f, server=jenkins-hbase4.apache.org,41201,1689422157218}] 2023-07-15 11:56:02,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:56:02,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-15 11:56:02,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41201%2C1689422157218.jenkins-hbase4.apache.org%2C41201%2C1689422157218.regiongroup-1, suffix=, logDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,41201,1689422157218, archiveDir=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs, maxLogs=10 2023-07-15 11:56:02,314 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:56:02,315 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => faf8bf9c0e6d77a0bdf0f3a70ab43734, NAME => 'hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:56:02,315 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:56:02,315 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:56:02,315 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:56:02,315 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:56:02,318 INFO [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:56:02,319 DEBUG [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/info 2023-07-15 11:56:02,319 DEBUG [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/info 2023-07-15 11:56:02,319 INFO [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region faf8bf9c0e6d77a0bdf0f3a70ab43734 columnFamilyName info 2023-07-15 11:56:02,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,41201,1689422157218/jenkins-hbase4.apache.org%2C41201%2C1689422157218.jenkins-hbase4.apache.org%2C41201%2C1689422157218.regiongroup-1.1689422162314 2023-07-15 11:56:02,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33417,DS-274df9f8-c308-48fc-8592-c230a3934cdd,DISK], DatanodeInfoWithStorage[127.0.0.1:45491,DS-f1a7c77d-1665-438f-b881-6671c08d57e3,DISK], DatanodeInfoWithStorage[127.0.0.1:39035,DS-50ad537a-587c-429a-875c-bc84b44b93df,DISK]] 2023-07-15 11:56:02,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ed49ae2049e592f2f5d8017bfcee4d2f, NAME => 't1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:56:02,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:56:02,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:56:02,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:56:02,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:56:02,326 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:56:02,326 DEBUG [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/info/06116a296dcb48c687c514a4eb1cd99f 2023-07-15 11:56:02,326 INFO [StoreOpener-faf8bf9c0e6d77a0bdf0f3a70ab43734-1] regionserver.HStore(310): Store=faf8bf9c0e6d77a0bdf0f3a70ab43734/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:56:02,327 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1 2023-07-15 11:56:02,327 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1 2023-07-15 11:56:02,327 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:56:02,327 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed49ae2049e592f2f5d8017bfcee4d2f columnFamilyName cf1 2023-07-15 11:56:02,328 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:56:02,331 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:56:02,332 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened faf8bf9c0e6d77a0bdf0f3a70ab43734; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70433570, jitterRate=0.049541980028152466}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-15 11:56:02,332 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for faf8bf9c0e6d77a0bdf0f3a70ab43734: 2023-07-15 11:56:02,334 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 451bdab9c3e24511a03375f24cd75048 2023-07-15 11:56:02,334 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1/451bdab9c3e24511a03375f24cd75048 2023-07-15 11:56:02,334 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734., pid=34, masterSystemTime=1689422162308 2023-07-15 11:56:02,337 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:56:02,337 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:56:02,337 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=faf8bf9c0e6d77a0bdf0f3a70ab43734, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:02,338 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689422162337"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422162337"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422162337"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422162337"}]},"ts":"1689422162337"} 2023-07-15 11:56:02,339 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1/d39c9fd504e448d9afb41abb769aeb63 2023-07-15 11:56:02,339 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(310): Store=ed49ae2049e592f2f5d8017bfcee4d2f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:56:02,339 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:56:02,340 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/norep 2023-07-15 11:56:02,340 DEBUG [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/norep 2023-07-15 11:56:02,341 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed49ae2049e592f2f5d8017bfcee4d2f columnFamilyName norep 2023-07-15 11:56:02,341 INFO [StoreOpener-ed49ae2049e592f2f5d8017bfcee4d2f-1] regionserver.HStore(310): Store=ed49ae2049e592f2f5d8017bfcee4d2f/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:56:02,342 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-07-15 11:56:02,342 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure faf8bf9c0e6d77a0bdf0f3a70ab43734, server=jenkins-hbase4.apache.org,41201,1689422157218 in 184 msec 2023-07-15 11:56:02,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:56:02,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:56:02,343 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=faf8bf9c0e6d77a0bdf0f3a70ab43734, ASSIGN in 350 msec 2023-07-15 11:56:02,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-15 11:56:02,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:56:02,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ed49ae2049e592f2f5d8017bfcee4d2f; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70187485, jitterRate=0.045875027775764465}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-15 11:56:02,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ed49ae2049e592f2f5d8017bfcee4d2f: 2023-07-15 11:56:02,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f., pid=35, masterSystemTime=1689422162308 2023-07-15 11:56:02,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:56:02,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:56:02,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:56:02,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8bb3eb49d5c769961a9b0f7da157f1d1, NAME => 't2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.', STARTKEY => '', ENDKEY => ''} 2023-07-15 11:56:02,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:56:02,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-15 11:56:02,350 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=ed49ae2049e592f2f5d8017bfcee4d2f, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:02,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:56:02,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:56:02,350 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422162350"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422162350"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422162350"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422162350"}]},"ts":"1689422162350"} 2023-07-15 11:56:02,351 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:56:02,352 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1 2023-07-15 11:56:02,352 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1 2023-07-15 11:56:02,353 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8bb3eb49d5c769961a9b0f7da157f1d1 columnFamilyName cf1 2023-07-15 11:56:02,354 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=31 2023-07-15 11:56:02,354 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=31, state=SUCCESS; OpenRegionProcedure ed49ae2049e592f2f5d8017bfcee4d2f, server=jenkins-hbase4.apache.org,41201,1689422157218 in 197 msec 2023-07-15 11:56:02,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=ed49ae2049e592f2f5d8017bfcee4d2f, ASSIGN in 362 msec 2023-07-15 11:56:02,359 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5cfa9235cb204ad1917d694f48598c17 2023-07-15 11:56:02,359 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1/5cfa9235cb204ad1917d694f48598c17 2023-07-15 11:56:02,362 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(539): loaded hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1/b3a999ddc2d147879011cf288ac4fcc0 2023-07-15 11:56:02,363 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(310): Store=8bb3eb49d5c769961a9b0f7da157f1d1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:56:02,363 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:56:02,363 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/norep 2023-07-15 11:56:02,364 DEBUG [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/norep 2023-07-15 11:56:02,364 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8bb3eb49d5c769961a9b0f7da157f1d1 columnFamilyName norep 2023-07-15 11:56:02,364 INFO [StoreOpener-8bb3eb49d5c769961a9b0f7da157f1d1-1] regionserver.HStore(310): Store=8bb3eb49d5c769961a9b0f7da157f1d1/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-15 11:56:02,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:56:02,366 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:56:02,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-15 11:56:02,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:56:02,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8bb3eb49d5c769961a9b0f7da157f1d1; next sequenceid=20; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67729532, jitterRate=0.009248673915863037}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-15 11:56:02,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8bb3eb49d5c769961a9b0f7da157f1d1: 2023-07-15 11:56:02,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1., pid=33, masterSystemTime=1689422162308 2023-07-15 11:56:02,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:56:02,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:56:02,372 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=8bb3eb49d5c769961a9b0f7da157f1d1, regionState=OPEN, openSeqNum=20, regionLocation=jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:02,372 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689422162372"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689422162372"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689422162372"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689422162372"}]},"ts":"1689422162372"} 2023-07-15 11:56:02,375 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=30 2023-07-15 11:56:02,375 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=30, state=SUCCESS; OpenRegionProcedure 8bb3eb49d5c769961a9b0f7da157f1d1, server=jenkins-hbase4.apache.org,41201,1689422157218 in 219 msec 2023-07-15 11:56:02,378 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-07-15 11:56:02,378 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,33403,1689422132751 after splitting done 2023-07-15 11:56:02,378 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=8bb3eb49d5c769961a9b0f7da157f1d1, ASSIGN in 383 msec 2023-07-15 11:56:02,378 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,33403,1689422132751 from processing; numProcessing=0 2023-07-15 11:56:02,380 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33403,1689422132751, splitWal=true, meta=true in 4.9910 sec 2023-07-15 11:56:02,523 INFO [jenkins-hbase4:41201Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:56:02,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-15 11:56:02,997 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-15 11:56:03,000 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-15 11:56:03,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.727sec 2023-07-15 11:56:03,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-15 11:56:03,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-15 11:56:03,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-15 11:56:03,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39715,1689422157186-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-15 11:56:03,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39715,1689422157186-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-15 11:56:03,003 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-15 11:56:03,059 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(139): Connect 0x34d33629 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:56:03,063 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69c030b6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:56:03,065 DEBUG [hconnection-0x62e633ab-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:56:03,066 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50372, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:56:03,069 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-15 11:56:03,069 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x34d33629 to 127.0.0.1:57666 2023-07-15 11:56:03,069 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:56:03,070 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:39715 after: jenkins-hbase4.apache.org:39715 2023-07-15 11:56:03,070 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(139): Connect 0x2d4722d9 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:56:03,074 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e6f37b1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:56:03,492 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-15 11:56:03,493 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-15 11:56:03,494 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-15 11:56:03,494 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-15 11:56:03,576 DEBUG [hconnection-0x39fd9b1b-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:56:03,578 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50376, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-15 11:56:03,604 INFO [Listener at localhost/46025] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689422163604 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:56:03,608 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): syncupReplication16894221636040x0, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:56:03,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689422163604-0x10168d630520036 connected Start Replication Server start 2023-07-15 11:56:03,611 INFO [Listener at localhost/46025] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-15 11:56:03,611 INFO [Listener at localhost/46025] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-15 11:56:03,629 INFO [Listener at localhost/46025] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-15 11:56:03,632 DEBUG [Listener at localhost/46025] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-15 11:56:03,648 INFO [Listener at localhost/46025] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-15 11:56:03,652 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,46571,1689422149499] other RSs: [] 2023-07-15 11:56:03,673 DEBUG [Listener at localhost/46025.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x14fa0091 to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:56:03,687 DEBUG [Listener at localhost/46025.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@a1fe16d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:56:03,687 INFO [Listener at localhost/46025.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:56:03,692 DEBUG [Listener at localhost/46025.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:56:03,693 INFO [Listener at localhost/46025.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=f0521d31-92f9-4caa-8fd4-4b1a00343aa2 to cluster=e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:56:03,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10168d630520038 connected 2023-07-15 11:56:06,789 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,46571,1689422149499/1's WALs to 1689422163611.syncuptool.replication.org,1234,1 2023-07-15 11:56:06,791 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1.1689422154536 with data PBUF\x08\xB7\x06 2023-07-15 11:56:06,792 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0.1689422149924 with data PBUF\x08\xD8\x05 2023-07-15 11:56:06,796 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,46571,1689422149499/1's WALs to 1689422163611.syncuptool.replication.org,1234,1 2023-07-15 11:56:07,523 INFO [jenkins-hbase4:41201Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:56:07,818 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-15 11:56:08,797 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46571,1689422149499] 2023-07-15 11:56:08,798 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46571,1689422149499] 2023-07-15 11:56:08,808 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46571,1689422149499] 2023-07-15 11:56:08,808 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,46571,1689422149499, ReplicationSource: 1, currentBandwidth=0 2023-07-15 11:56:08,808 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:56:08,827 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499] zookeeper.ReadOnlyZKClient(139): Connect 0x43593bac to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:56:08,836 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3e7338c6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:56:08,836 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:57666 2023-07-15 11:56:08,838 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-15 11:56:08,839 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10168d63052003a connected 2023-07-15 11:56:08,839 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,46571,1689422149499 (queues=2) is replicating from cluster=f0521d31-92f9-4caa-8fd4-4b1a00343aa2 to cluster=e23fdc13-766c-4283-8b71-9a14496acb3e 2023-07-15 11:56:08,840 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0 2023-07-15 11:56:08,841 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,46571,1689422149499, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-15 11:56:08,842 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1 2023-07-15 11:56:08,842 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0 2023-07-15 11:56:08,843 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,46571,1689422149499, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-15 11:56:08,844 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1 2023-07-15 11:56:08,848 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0.1689422149924 2023-07-15 11:56:08,848 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-15 11:56:08,849 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,46571,1689422149499 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-15 11:56:08,849 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:56:08,854 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1.1689422154536 2023-07-15 11:56:08,854 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceWALReader(162): Read 300 WAL entries eligible for replication 2023-07-15 11:56:08,854 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-15 11:56:08,856 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,46571,1689422149499 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/eee02c0767aa2be4d83f125ea78097e3/112=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/113=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/114=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/115=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/116=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/117=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/118=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/119=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/120=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/121=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/122=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/123=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/124=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/125=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/126=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/127=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/128=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/129=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/130=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/131=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/132=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/133=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/134=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/135=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/136=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/137=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/138=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/139=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/140=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/141=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/142=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/143=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/144=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/145=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/146=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/147=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/148=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/149=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/150=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/151=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/152=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/153=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/154=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/155=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/156=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/157=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/158=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/159=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/160=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/161=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/162=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/163=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/164=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/165=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/166=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/167=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/168=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/169=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/170=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/171=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/172=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/173=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/174=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/175=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/176=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/177=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/178=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/179=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/180=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/181=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/182=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/183=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/184=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/185=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/186=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/187=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/188=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/189=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/190=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/191=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/192=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/193=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/194=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/195=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/196=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/197=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/198=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/199=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/200=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/201=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/202=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/203=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/204=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/205=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/206=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/207=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/208=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/209=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/210=[#edits: 1 = ],208}, {t1_syncup/eee02c0767aa2be4d83f125ea78097e3/211=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/212=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/213=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/214=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/215=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/216=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/217=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/218=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/219=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/220=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/221=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/222=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/223=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/224=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/225=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/226=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/227=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/228=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/229=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/230=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/231=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/232=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/233=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/234=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/235=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/236=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/237=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/238=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/239=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/240=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/241=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/242=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/243=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/244=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/245=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/246=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/247=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/248=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/249=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/250=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/251=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/252=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/253=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/254=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/255=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/256=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/257=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/258=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/259=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/260=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/261=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/262=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/263=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/264=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/265=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/266=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/267=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/268=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/269=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/270=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/271=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/272=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/273=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/274=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/275=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/276=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/277=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/278=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/279=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/280=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/281=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/282=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/283=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/284=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/285=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/286=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/287=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/288=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/289=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/290=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/291=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/292=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/293=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/294=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/295=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/296=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/297=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/298=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/299=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/300=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/301=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/302=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/303=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/304=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/305=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/306=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/307=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/308=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/309=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/310=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/311=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/312=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/313=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/314=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/315=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/316=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/317=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/318=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/319=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/320=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/321=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/322=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/323=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/324=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/325=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/326=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/327=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/328=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/329=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/330=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/331=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/332=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/333=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/334=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/335=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/336=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/337=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/338=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/339=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/340=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/341=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/342=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/343=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/344=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/345=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/346=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/347=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/348=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/349=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/350=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/351=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/352=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/353=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/354=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/355=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/356=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/357=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/358=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/359=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/360=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/361=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/362=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/363=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/364=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/365=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/366=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/367=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/368=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/369=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/370=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/371=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/372=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/373=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/374=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/375=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/376=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/377=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/378=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/379=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/380=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/381=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/382=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/383=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/384=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/385=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/386=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/387=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/388=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/389=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/390=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/391=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/392=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/393=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/394=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/395=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/396=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/397=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/398=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/399=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/400=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/401=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/402=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/403=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/404=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/405=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/406=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/407=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/408=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/409=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/410=[#edits: 1 = ],208}, {t2_syncup/89b547022cb2e9610cb6246270f3cbaf/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:44973/user/jenkins/test-data/6e73984d-0013-6d0f-d416-ece2173dfcc1/oldWALs/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1.1689422154536, lastWalPosition=34707, nbRowKeys=300, nbHFiles=0, heapSize=62400, lastSeqIds={}, endOfFile=true,usedBufferSize=62400] 2023-07-15 11:56:08,862 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-15 11:56:08,865 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50386, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-15 11:56:08,869 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41201] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-15 11:56:08,870 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41201] zookeeper.ReadOnlyZKClient(139): Connect 0x26b101fd to 127.0.0.1:57666 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-15 11:56:08,873 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41201] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@2876eb5b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-15 11:56:08,875 DEBUG [hconnection-0x2c496a8d-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-15 11:56:08,878 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50402, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-07-15 11:56:08,897 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41201] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-15 11:56:08,898 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46571,1689422149499] 2023-07-15 11:56:08,899 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1.1689422154536] 2023-07-15 11:56:08,907 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,46571,1689422149499 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-15 11:56:08,907 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:56:08,949 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:56:08,950 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1-jenkins-hbase4.apache.org,46571,1689422149499] zookeeper.ZKUtil(162): syncupReplication1689422163604-0x10168d630520036, quorum=127.0.0.1:57666, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1689422163611.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0.1689422149924 2023-07-15 11:56:08,952 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689422163604-0x10168d630520036, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689422163611.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46571,1689422149499/jenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0.1689422149924 2023-07-15 11:56:08,952 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689422163604-0x10168d630520036, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1689422163611.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:56:08,952 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689422163604-0x10168d630520036, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689422163611.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:56:08,952 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-0,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,46571,1689422149499 with the following stats: Total replicated edits: 300, current progress: 2023-07-15 11:56:09,053 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,46571,1689422149499 2023-07-15 11:56:09,053 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] zookeeper.ZKUtil(243): syncupReplication1689422163604-0x10168d630520036, quorum=127.0.0.1:57666, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689422163611.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46571,1689422149499 because node does not exist (not an error) 2023-07-15 11:56:09,054 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499.replicationSource.shipperjenkins-hbase4.apache.org%2C46571%2C1689422149499.jenkins-hbase4.apache.org%2C46571%2C1689422149499.regiongroup-1,1-jenkins-hbase4.apache.org,46571,1689422149499] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,46571,1689422149499 with the following stats: Total replicated edits: 300, current progress: 2023-07-15 11:56:12,523 INFO [jenkins-hbase4:41201Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-15 11:56:12,523 INFO [jenkins-hbase4:41201Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-07-15 11:56:13,652 INFO [Listener at localhost/46025] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-15 11:56:13,757 DEBUG [Listener at localhost/46025.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520038, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:56:13,757 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x14fa0091 to 127.0.0.1:57666 2023-07-15 11:56:13,759 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:56:13,961 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689422163604-0x10168d630520036, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:56:13,962 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689422163604-0x10168d630520036, quorum=127.0.0.1:57666, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:56:13,970 INFO [Listener at localhost/46025] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-07-15 11:56:13,970 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-15 11:56:13,970 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2d4722d9 to 127.0.0.1:57666 2023-07-15 11:56:13,971 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:56:13,971 DEBUG [Listener at localhost/46025] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-15 11:56:13,971 DEBUG [Listener at localhost/46025] util.JVMClusterUtil(257): Found active master hash=988410583, stopped=false 2023-07-15 11:56:13,971 INFO [Listener at localhost/46025] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:56:13,974 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:41201-0x10168d630520031, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-15 11:56:13,974 INFO [Listener at localhost/46025] procedure2.ProcedureExecutor(629): Stopping 2023-07-15 11:56:13,974 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-15 11:56:13,974 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:56:13,974 DEBUG [Listener at localhost/46025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4eb9c718 to 127.0.0.1:57666 2023-07-15 11:56:13,979 DEBUG [Listener at localhost/46025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:56:13,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41201-0x10168d630520031, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:56:13,979 INFO [Listener at localhost/46025] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41201,1689422157218' ***** 2023-07-15 11:56:13,979 INFO [Listener at localhost/46025] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-15 11:56:13,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-15 11:56:13,979 INFO [RS:0;jenkins-hbase4:41201] regionserver.HeapMemoryManager(220): Stopping 2023-07-15 11:56:13,979 INFO [RS:0;jenkins-hbase4:41201] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-15 11:56:13,979 INFO [RS:0;jenkins-hbase4:41201] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-15 11:56:13,980 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(3305): Received CLOSE for ed49ae2049e592f2f5d8017bfcee4d2f 2023-07-15 11:56:13,980 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-15 11:56:13,981 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(3305): Received CLOSE for 8bb3eb49d5c769961a9b0f7da157f1d1 2023-07-15 11:56:13,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ed49ae2049e592f2f5d8017bfcee4d2f, disabling compactions & flushes 2023-07-15 11:56:13,981 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(3305): Received CLOSE for faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:56:13,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:56:13,981 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:13,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:56:13,982 DEBUG [RS:0;jenkins-hbase4:41201] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x547904a8 to 127.0.0.1:57666 2023-07-15 11:56:13,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. after waiting 0 ms 2023-07-15 11:56:13,982 DEBUG [RS:0;jenkins-hbase4:41201] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:56:13,982 INFO [RS:0;jenkins-hbase4:41201] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-15 11:56:13,982 INFO [RS:0;jenkins-hbase4:41201] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-15 11:56:13,982 INFO [RS:0;jenkins-hbase4:41201] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-15 11:56:13,982 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-15 11:56:13,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:56:13,982 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-15 11:56:13,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ed49ae2049e592f2f5d8017bfcee4d2f 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-15 11:56:13,983 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-15 11:56:13,983 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-15 11:56:13,983 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-15 11:56:13,983 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-15 11:56:13,983 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-15 11:56:13,983 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-15 11:56:13,982 DEBUG [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1478): Online Regions={ed49ae2049e592f2f5d8017bfcee4d2f=t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f., 1588230740=hbase:meta,,1.1588230740, 8bb3eb49d5c769961a9b0f7da157f1d1=t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1., faf8bf9c0e6d77a0bdf0f3a70ab43734=hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734.} 2023-07-15 11:56:13,984 DEBUG [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1504): Waiting on 1588230740, 8bb3eb49d5c769961a9b0f7da157f1d1, ed49ae2049e592f2f5d8017bfcee4d2f, faf8bf9c0e6d77a0bdf0f3a70ab43734 2023-07-15 11:56:14,011 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/.tmp/cf1/568d69de4be040a9b6a4fa2f28d86e16 2023-07-15 11:56:14,014 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:56:14,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/.tmp/cf1/568d69de4be040a9b6a4fa2f28d86e16 as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1/568d69de4be040a9b6a4fa2f28d86e16 2023-07-15 11:56:14,023 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/.tmp/info/7f954cccceeb43e495d5d9333be48b51 2023-07-15 11:56:14,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/cf1/568d69de4be040a9b6a4fa2f28d86e16, entries=100, sequenceid=21, filesize=8.3 K 2023-07-15 11:56:14,029 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/.tmp/info/7f954cccceeb43e495d5d9333be48b51 as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info/7f954cccceeb43e495d5d9333be48b51 2023-07-15 11:56:14,030 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for ed49ae2049e592f2f5d8017bfcee4d2f in 48ms, sequenceid=21, compaction requested=true 2023-07-15 11:56:14,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t1_syncup/ed49ae2049e592f2f5d8017bfcee4d2f/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-07-15 11:56:14,041 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/info/7f954cccceeb43e495d5d9333be48b51, entries=24, sequenceid=43, filesize=7.4 K 2023-07-15 11:56:14,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:56:14,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ed49ae2049e592f2f5d8017bfcee4d2f: 2023-07-15 11:56:14,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689422126786.ed49ae2049e592f2f5d8017bfcee4d2f. 2023-07-15 11:56:14,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8bb3eb49d5c769961a9b0f7da157f1d1, disabling compactions & flushes 2023-07-15 11:56:14,043 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:56:14,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:56:14,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. after waiting 0 ms 2023-07-15 11:56:14,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:56:14,043 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8bb3eb49d5c769961a9b0f7da157f1d1 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-15 11:56:14,045 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 62ms, sequenceid=43, compaction requested=true 2023-07-15 11:56:14,057 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-07-15 11:56:14,057 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-15 11:56:14,058 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-15 11:56:14,058 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-15 11:56:14,058 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-15 11:56:14,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=23 (bloomFilter=true), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/.tmp/cf1/7c29254439354016b6575b63482c81aa 2023-07-15 11:56:14,075 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/.tmp/cf1/7c29254439354016b6575b63482c81aa as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1/7c29254439354016b6575b63482c81aa 2023-07-15 11:56:14,080 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/cf1/7c29254439354016b6575b63482c81aa, entries=200, sequenceid=23, filesize=12.2 K 2023-07-15 11:56:14,081 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 8bb3eb49d5c769961a9b0f7da157f1d1 in 38ms, sequenceid=23, compaction requested=true 2023-07-15 11:56:14,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/default/t2_syncup/8bb3eb49d5c769961a9b0f7da157f1d1/recovered.edits/26.seqid, newMaxSeqId=26, maxSeqId=19 2023-07-15 11:56:14,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:56:14,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8bb3eb49d5c769961a9b0f7da157f1d1: 2023-07-15 11:56:14,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689422127403.8bb3eb49d5c769961a9b0f7da157f1d1. 2023-07-15 11:56:14,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing faf8bf9c0e6d77a0bdf0f3a70ab43734, disabling compactions & flushes 2023-07-15 11:56:14,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:56:14,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:56:14,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. after waiting 0 ms 2023-07-15 11:56:14,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:56:14,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/data/hbase/namespace/faf8bf9c0e6d77a0bdf0f3a70ab43734/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-07-15 11:56:14,101 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:56:14,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for faf8bf9c0e6d77a0bdf0f3a70ab43734: 2023-07-15 11:56:14,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689422124830.faf8bf9c0e6d77a0bdf0f3a70ab43734. 2023-07-15 11:56:14,184 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41201,1689422157218; all regions closed. 2023-07-15 11:56:14,184 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:14,192 DEBUG [RS:0;jenkins-hbase4:41201] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:56:14,192 INFO [RS:0;jenkins-hbase4:41201] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C41201%2C1689422157218.meta:.meta(num 1689422157767) 2023-07-15 11:56:14,192 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:14,199 DEBUG [RS:0;jenkins-hbase4:41201] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:56:14,199 INFO [RS:0;jenkins-hbase4:41201] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C41201%2C1689422157218.jenkins-hbase4.apache.org%2C41201%2C1689422157218.regiongroup-0:(num 1689422157628) 2023-07-15 11:56:14,199 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/WALs/jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:14,207 DEBUG [RS:0;jenkins-hbase4:41201] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/oldWALs 2023-07-15 11:56:14,207 INFO [RS:0;jenkins-hbase4:41201] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C41201%2C1689422157218.jenkins-hbase4.apache.org%2C41201%2C1689422157218.regiongroup-1:(num 1689422162314) 2023-07-15 11:56:14,207 DEBUG [RS:0;jenkins-hbase4:41201] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:56:14,207 INFO [RS:0;jenkins-hbase4:41201] regionserver.LeaseManager(133): Closed leases 2023-07-15 11:56:14,207 INFO [RS:0;jenkins-hbase4:41201] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-15 11:56:14,207 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:56:14,207 DEBUG [RS:0;jenkins-hbase4:41201] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x26b101fd to 127.0.0.1:57666 2023-07-15 11:56:14,207 DEBUG [RS:0;jenkins-hbase4:41201] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:56:14,208 INFO [RS:0;jenkins-hbase4:41201] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41201 2023-07-15 11:56:14,212 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:56:14,212 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d630520026, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:56:14,213 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:41201-0x10168d630520031, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,41201,1689422157218 2023-07-15 11:56:14,213 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10168d63052003a, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:56:14,213 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:41201-0x10168d630520031, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-15 11:56:14,213 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-15 11:56:14,213 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-15 11:56:14,213 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41201,1689422157218] 2023-07-15 11:56:14,213 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41201,1689422157218; numProcessing=1 2023-07-15 11:56:14,214 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,41201,1689422157218 already deleted, retry=false 2023-07-15 11:56:14,214 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41201,1689422157218 expired; onlineServers=0 2023-07-15 11:56:14,214 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39715,1689422157186' ***** 2023-07-15 11:56:14,214 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-15 11:56:14,215 DEBUG [M:0;jenkins-hbase4:39715] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3ed66152, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-15 11:56:14,215 INFO [M:0;jenkins-hbase4:39715] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:56:14,215 INFO [M:0;jenkins-hbase4:39715] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39715,1689422157186; all regions closed. 2023-07-15 11:56:14,215 DEBUG [M:0;jenkins-hbase4:39715] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-15 11:56:14,215 DEBUG [M:0;jenkins-hbase4:39715] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-15 11:56:14,215 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-15 11:56:14,215 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422157440] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689422157440,5,FailOnTimeoutGroup] 2023-07-15 11:56:14,215 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422157440] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689422157440,5,FailOnTimeoutGroup] 2023-07-15 11:56:14,215 DEBUG [M:0;jenkins-hbase4:39715] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-15 11:56:14,217 INFO [M:0;jenkins-hbase4:39715] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-15 11:56:14,217 INFO [M:0;jenkins-hbase4:39715] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-15 11:56:14,217 INFO [M:0;jenkins-hbase4:39715] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-15 11:56:14,217 DEBUG [M:0;jenkins-hbase4:39715] master.HMaster(1512): Stopping service threads 2023-07-15 11:56:14,217 INFO [M:0;jenkins-hbase4:39715] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-15 11:56:14,218 ERROR [M:0;jenkins-hbase4:39715] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-15 11:56:14,218 INFO [M:0;jenkins-hbase4:39715] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-15 11:56:14,218 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-15 11:56:14,218 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-15 11:56:14,218 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-15 11:56:14,219 DEBUG [M:0;jenkins-hbase4:39715] zookeeper.ZKUtil(398): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-15 11:56:14,219 WARN [M:0;jenkins-hbase4:39715] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-15 11:56:14,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-15 11:56:14,219 INFO [M:0;jenkins-hbase4:39715] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-15 11:56:14,219 INFO [M:0;jenkins-hbase4:39715] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-15 11:56:14,220 DEBUG [M:0;jenkins-hbase4:39715] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-15 11:56:14,220 INFO [M:0;jenkins-hbase4:39715] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:56:14,220 DEBUG [M:0;jenkins-hbase4:39715] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:56:14,220 DEBUG [M:0;jenkins-hbase4:39715] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-15 11:56:14,220 DEBUG [M:0;jenkins-hbase4:39715] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:56:14,220 INFO [M:0;jenkins-hbase4:39715] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-07-15 11:56:14,232 INFO [M:0;jenkins-hbase4:39715] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=290 (bloomFilter=true), to=hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/50980fc08e9f4abfab06582297879753 2023-07-15 11:56:14,237 DEBUG [M:0;jenkins-hbase4:39715] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/50980fc08e9f4abfab06582297879753 as hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/50980fc08e9f4abfab06582297879753 2023-07-15 11:56:14,242 INFO [M:0;jenkins-hbase4:39715] regionserver.HStore(1080): Added hdfs://localhost:46809/user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/50980fc08e9f4abfab06582297879753, entries=9, sequenceid=290, filesize=5.5 K 2023-07-15 11:56:14,243 INFO [M:0;jenkins-hbase4:39715] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=290, compaction requested=true 2023-07-15 11:56:14,243 INFO [M:0;jenkins-hbase4:39715] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-15 11:56:14,243 DEBUG [M:0;jenkins-hbase4:39715] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-15 11:56:14,244 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/e4e3bb03-049b-affa-f218-ce8573544346/MasterData/WALs/jenkins-hbase4.apache.org,39715,1689422157186 2023-07-15 11:56:14,251 INFO [M:0;jenkins-hbase4:39715] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-15 11:56:14,251 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-15 11:56:14,252 INFO [M:0;jenkins-hbase4:39715] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39715 2023-07-15 11:56:14,255 DEBUG [M:0;jenkins-hbase4:39715] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,39715,1689422157186 already deleted, retry=false 2023-07-15 11:56:14,374 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:56:14,374 INFO [M:0;jenkins-hbase4:39715] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39715,1689422157186; zookeeper connection closed. 2023-07-15 11:56:14,374 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): master:39715-0x10168d630520030, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:56:14,453 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-15 11:56:14,474 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:41201-0x10168d630520031, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:56:14,474 INFO [RS:0;jenkins-hbase4:41201] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41201,1689422157218; zookeeper connection closed. 2023-07-15 11:56:14,474 DEBUG [Listener at localhost/46025-EventThread] zookeeper.ZKWatcher(600): regionserver:41201-0x10168d630520031, quorum=127.0.0.1:57666, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-15 11:56:14,476 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2e4bec81] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2e4bec81 2023-07-15 11:56:14,476 INFO [Listener at localhost/46025] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-15 11:56:14,476 WARN [Listener at localhost/46025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-15 11:56:14,483 INFO [Listener at localhost/46025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-15 11:56:14,589 WARN [BP-849328800-172.31.14.131-1689422122581 heartbeating to localhost/127.0.0.1:46809] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-15 11:56:14,589 WARN [BP-849328800-172.31.14.131-1689422122581 heartbeating to localhost/127.0.0.1:46809] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-849328800-172.31.14.131-1689422122581 (Datanode Uuid a69a4743-a0c3-4c29-9dda-d3423c7addd2) service to localhost/127.0.0.1:46809 2023-07-15 11:56:14,591 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/cluster_a703bc18-c99b-2972-822e-fef9de60bd53/dfs/data/data7/current/BP-849328800-172.31.14.131-1689422122581] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:14,591 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/cluster_a703bc18-c99b-2972-822e-fef9de60bd53/dfs/data/data8/current/BP-849328800-172.31.14.131-1689422122581] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:14,593 WARN [Listener at localhost/46025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-15 11:56:14,596 INFO [Listener at localhost/46025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-15 11:56:14,703 WARN [BP-849328800-172.31.14.131-1689422122581 heartbeating to localhost/127.0.0.1:46809] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-15 11:56:14,703 WARN [BP-849328800-172.31.14.131-1689422122581 heartbeating to localhost/127.0.0.1:46809] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-849328800-172.31.14.131-1689422122581 (Datanode Uuid cdca35cf-6686-4a6c-a077-f1918b45e3da) service to localhost/127.0.0.1:46809 2023-07-15 11:56:14,703 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/cluster_a703bc18-c99b-2972-822e-fef9de60bd53/dfs/data/data5/current/BP-849328800-172.31.14.131-1689422122581] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:14,703 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/cluster_a703bc18-c99b-2972-822e-fef9de60bd53/dfs/data/data6/current/BP-849328800-172.31.14.131-1689422122581] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:14,705 WARN [Listener at localhost/46025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-15 11:56:14,708 INFO [Listener at localhost/46025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-15 11:56:14,812 WARN [BP-849328800-172.31.14.131-1689422122581 heartbeating to localhost/127.0.0.1:46809] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-15 11:56:14,812 WARN [BP-849328800-172.31.14.131-1689422122581 heartbeating to localhost/127.0.0.1:46809] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-849328800-172.31.14.131-1689422122581 (Datanode Uuid 0c0b17eb-6c29-4bb5-bc81-bfa1e5e20d5c) service to localhost/127.0.0.1:46809 2023-07-15 11:56:14,813 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/cluster_a703bc18-c99b-2972-822e-fef9de60bd53/dfs/data/data3/current/BP-849328800-172.31.14.131-1689422122581] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:14,813 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/cluster_a703bc18-c99b-2972-822e-fef9de60bd53/dfs/data/data4/current/BP-849328800-172.31.14.131-1689422122581] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:14,815 WARN [Listener at localhost/46025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-15 11:56:14,818 INFO [Listener at localhost/46025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-15 11:56:14,923 WARN [BP-849328800-172.31.14.131-1689422122581 heartbeating to localhost/127.0.0.1:46809] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-15 11:56:14,923 WARN [BP-849328800-172.31.14.131-1689422122581 heartbeating to localhost/127.0.0.1:46809] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-849328800-172.31.14.131-1689422122581 (Datanode Uuid 10650aa3-cc51-41e1-84ee-1e33ce8c7eaf) service to localhost/127.0.0.1:46809 2023-07-15 11:56:14,924 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/cluster_a703bc18-c99b-2972-822e-fef9de60bd53/dfs/data/data1/current/BP-849328800-172.31.14.131-1689422122581] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:14,924 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/76f5b564-d491-25bf-21a4-a042dfea517b/cluster_a703bc18-c99b-2972-822e-fef9de60bd53/dfs/data/data2/current/BP-849328800-172.31.14.131-1689422122581] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:14,953 INFO [Listener at localhost/46025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-15 11:56:15,137 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-15 11:56:15,138 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-15 11:56:15,138 WARN [Listener at localhost/46025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-15 11:56:15,140 INFO [Listener at localhost/46025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-15 11:56:15,241 WARN [BP-2129747630-172.31.14.131-1689422115308 heartbeating to localhost/127.0.0.1:44973] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-15 11:56:15,241 WARN [BP-2129747630-172.31.14.131-1689422115308 heartbeating to localhost/127.0.0.1:44973] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2129747630-172.31.14.131-1689422115308 (Datanode Uuid 1840765f-84f0-4d5b-b6a6-aff031f617fa) service to localhost/127.0.0.1:44973 2023-07-15 11:56:15,241 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/cluster_ccea1b3f-c80a-2107-12f6-5d4a658e0622/dfs/data/data3/current/BP-2129747630-172.31.14.131-1689422115308] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:15,242 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/cluster_ccea1b3f-c80a-2107-12f6-5d4a658e0622/dfs/data/data4/current/BP-2129747630-172.31.14.131-1689422115308] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:15,242 WARN [Listener at localhost/46025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-15 11:56:15,244 INFO [Listener at localhost/46025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-15 11:56:15,345 WARN [BP-2129747630-172.31.14.131-1689422115308 heartbeating to localhost/127.0.0.1:44973] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-15 11:56:15,345 WARN [BP-2129747630-172.31.14.131-1689422115308 heartbeating to localhost/127.0.0.1:44973] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2129747630-172.31.14.131-1689422115308 (Datanode Uuid 9cf91406-fb7f-48b9-8af1-5c2d38b174cd) service to localhost/127.0.0.1:44973 2023-07-15 11:56:15,346 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/cluster_ccea1b3f-c80a-2107-12f6-5d4a658e0622/dfs/data/data1/current/BP-2129747630-172.31.14.131-1689422115308] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:15,346 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c660979c-f0f2-9b3d-e73d-233e132b6471/cluster_ccea1b3f-c80a-2107-12f6-5d4a658e0622/dfs/data/data2/current/BP-2129747630-172.31.14.131-1689422115308] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-15 11:56:15,354 INFO [Listener at localhost/46025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-15 11:56:15,469 INFO [Listener at localhost/46025] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-15 11:56:15,494 INFO [Listener at localhost/46025] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-15 11:56:15,517 INFO [Listener at localhost/46025] hbase.ResourceChecker(175): after: replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL#testSyncUpTool Thread=183 (was 10) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x43593bac-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x3aa4b160 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/2063010540.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:46809 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x52264cd1-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:46809 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x52264cd1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/2063010540.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:44973 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x43593bac-SendThread(127.0.0.1:57666) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1542168714) connection to localhost/127.0.0.1:46809 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:44973 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x3aa4b160-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:46809 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@67600b78 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:44973 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:46809 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1542168714) connection to localhost/127.0.0.1:44973 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1542168714) connection to localhost/127.0.0.1:46809 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: Listener at localhost/46025 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1542168714) connection to localhost/127.0.0.1:46809 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:46809 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x2a94bf46-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x52264cd1-SendThread(127.0.0.1:57666) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46571,1689422149499-SendThread(127.0.0.1:57666) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x2a94bf46-SendThread(127.0.0.1:57666) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x43593bac sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/2063010540.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x2a94bf46 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/2063010540.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57666@0x3aa4b160-SendThread(127.0.0.1:57666) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:44973 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:46809 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1542168714) connection to localhost/127.0.0.1:44973 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:46809 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34419,1689422119824-SendThread(127.0.0.1:57666) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) - Thread LEAK? -, OpenFileDescriptor=688 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=236 (was 215) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=5700 (was 7138)