2023-07-20 02:55:26,356 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be 2023-07-20 02:55:26,368 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-log-dir so I do NOT create it in target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42 2023-07-20 02:55:26,368 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-log-dir Erasing configuration value by system value. 2023-07-20 02:55:26,368 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-tmp-dir so I do NOT create it in target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42 2023-07-20 02:55:26,369 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-20 02:55:26,369 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42 2023-07-20 02:55:26,386 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-07-20 02:55:26,423 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.TestReplicationSyncUpTool#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=200, ProcessCount=171, AvailableMemoryMB=6947 2023-07-20 02:55:26,436 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-log-dir so I do NOT create it in target/test-data/901d358d-76ab-6ee3-5ba7-a875bc5ca02a 2023-07-20 02:55:26,436 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-log-dir Erasing configuration value by system value. 2023-07-20 02:55:26,436 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-tmp-dir so I do NOT create it in target/test-data/901d358d-76ab-6ee3-5ba7-a875bc5ca02a 2023-07-20 02:55:26,437 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-20 02:55:26,437 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/901d358d-76ab-6ee3-5ba7-a875bc5ca02a 2023-07-20 02:55:26,446 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-log-dir so I do NOT create it in target/test-data/55d80fc5-d1cd-2254-6e0e-d7fe33f0d7ed 2023-07-20 02:55:26,446 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-log-dir Erasing configuration value by system value. 2023-07-20 02:55:26,446 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-tmp-dir so I do NOT create it in target/test-data/55d80fc5-d1cd-2254-6e0e-d7fe33f0d7ed 2023-07-20 02:55:26,447 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-20 02:55:26,447 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/55d80fc5-d1cd-2254-6e0e-d7fe33f0d7ed 2023-07-20 02:55:26,452 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/cluster_5c0383fa-0b5a-b542-a270-52985b3db8f2, deleteOnExit=true 2023-07-20 02:55:26,600 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/cluster_5c0383fa-0b5a-b542-a270-52985b3db8f2/zookeeper_0, clientPort=55129, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/cluster_5c0383fa-0b5a-b542-a270-52985b3db8f2/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/cluster_5c0383fa-0b5a-b542-a270-52985b3db8f2/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-20 02:55:26,617 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55129 2023-07-20 02:55:26,621 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-20 02:55:26,621 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-20 02:55:26,621 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/test.cache.data in system properties and HBase conf 2023-07-20 02:55:26,622 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop.tmp.dir in system properties and HBase conf 2023-07-20 02:55:26,622 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/hadoop.log.dir in system properties and HBase conf 2023-07-20 02:55:26,623 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-20 02:55:26,623 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-20 02:55:26,623 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-20 02:55:26,730 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-20 02:55:27,121 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-20 02:55:27,125 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-20 02:55:27,126 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-20 02:55:27,126 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-20 02:55:27,126 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-20 02:55:27,127 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-20 02:55:27,127 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-20 02:55:27,127 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-20 02:55:27,127 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-20 02:55:27,128 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-20 02:55:27,128 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/nfs.dump.dir in system properties and HBase conf 2023-07-20 02:55:27,129 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/java.io.tmpdir in system properties and HBase conf 2023-07-20 02:55:27,129 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-20 02:55:27,129 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-20 02:55:27,130 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-20 02:55:27,679 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-20 02:55:27,683 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-20 02:55:27,972 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-20 02:55:28,103 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-20 02:55:28,124 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 02:55:28,168 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 02:55:28,206 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/java.io.tmpdir/Jetty_localhost_40859_hdfs____iqvkay/webapp 2023-07-20 02:55:28,358 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40859 2023-07-20 02:55:28,376 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-20 02:55:28,377 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-20 02:55:28,930 WARN [Listener at localhost/37763] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 02:55:29,016 WARN [Listener at localhost/37763] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 02:55:29,040 WARN [Listener at localhost/37763] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 02:55:29,071 INFO [Listener at localhost/37763] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 02:55:29,085 INFO [Listener at localhost/37763] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/java.io.tmpdir/Jetty_localhost_34185_datanode____wtup90/webapp 2023-07-20 02:55:29,213 INFO [Listener at localhost/37763] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34185 2023-07-20 02:55:29,520 WARN [Listener at localhost/42947] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 02:55:29,532 WARN [Listener at localhost/42947] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 02:55:29,534 WARN [Listener at localhost/42947] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 02:55:29,536 INFO [Listener at localhost/42947] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 02:55:29,541 INFO [Listener at localhost/42947] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/java.io.tmpdir/Jetty_localhost_36697_datanode____.jiv9xk/webapp 2023-07-20 02:55:29,643 INFO [Listener at localhost/42947] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36697 2023-07-20 02:55:29,657 WARN [Listener at localhost/35915] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 02:55:30,040 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3cbba31cef451374: Processing first storage report for DS-bb6f101f-116f-465d-8349-dd6666cb59ca from datanode 6b20068e-fbad-42bc-ad88-732810a3afce 2023-07-20 02:55:30,042 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3cbba31cef451374: from storage DS-bb6f101f-116f-465d-8349-dd6666cb59ca node DatanodeRegistration(127.0.0.1:40993, datanodeUuid=6b20068e-fbad-42bc-ad88-732810a3afce, infoPort=42397, infoSecurePort=0, ipcPort=35915, storageInfo=lv=-57;cid=testClusterID;nsid=198095513;c=1689821727757), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-20 02:55:30,042 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6a942287ae25b5ff: Processing first storage report for DS-3b140a78-8011-4c98-96ec-6684fce7ecc8 from datanode 3220ed50-8801-4122-923c-5b3408ae42d9 2023-07-20 02:55:30,043 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6a942287ae25b5ff: from storage DS-3b140a78-8011-4c98-96ec-6684fce7ecc8 node DatanodeRegistration(127.0.0.1:44997, datanodeUuid=3220ed50-8801-4122-923c-5b3408ae42d9, infoPort=34271, infoSecurePort=0, ipcPort=42947, storageInfo=lv=-57;cid=testClusterID;nsid=198095513;c=1689821727757), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-20 02:55:30,043 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3cbba31cef451374: Processing first storage report for DS-3961a4b7-9e20-438a-82a8-658ddbb7d3c4 from datanode 6b20068e-fbad-42bc-ad88-732810a3afce 2023-07-20 02:55:30,043 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3cbba31cef451374: from storage DS-3961a4b7-9e20-438a-82a8-658ddbb7d3c4 node DatanodeRegistration(127.0.0.1:40993, datanodeUuid=6b20068e-fbad-42bc-ad88-732810a3afce, infoPort=42397, infoSecurePort=0, ipcPort=35915, storageInfo=lv=-57;cid=testClusterID;nsid=198095513;c=1689821727757), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 02:55:30,043 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6a942287ae25b5ff: Processing first storage report for DS-c9638fdf-7887-46c1-8277-19b29e22d42a from datanode 3220ed50-8801-4122-923c-5b3408ae42d9 2023-07-20 02:55:30,043 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6a942287ae25b5ff: from storage DS-c9638fdf-7887-46c1-8277-19b29e22d42a node DatanodeRegistration(127.0.0.1:44997, datanodeUuid=3220ed50-8801-4122-923c-5b3408ae42d9, infoPort=34271, infoSecurePort=0, ipcPort=42947, storageInfo=lv=-57;cid=testClusterID;nsid=198095513;c=1689821727757), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 02:55:30,130 DEBUG [Listener at localhost/35915] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be 2023-07-20 02:55:30,140 INFO [Listener at localhost/35915] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:30,143 INFO [Listener at localhost/35915] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:30,865 INFO [Listener at localhost/35915] util.FSUtils(471): Created version file at hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61 with version=8 2023-07-20 02:55:30,865 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/hbase-staging 2023-07-20 02:55:31,205 INFO [Listener at localhost/35915] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-20 02:55:31,716 INFO [Listener at localhost/35915] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-20 02:55:31,747 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:31,748 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:31,748 INFO [Listener at localhost/35915] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:55:31,748 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:31,748 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:55:31,898 INFO [Listener at localhost/35915] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:55:31,981 DEBUG [Listener at localhost/35915] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-20 02:55:32,076 INFO [Listener at localhost/35915] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36235 2023-07-20 02:55:32,086 INFO [Listener at localhost/35915] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:32,088 INFO [Listener at localhost/35915] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:32,114 INFO [Listener at localhost/35915] zookeeper.RecoverableZooKeeper(93): Process identifier=master:36235 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:32,158 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:362350x0, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:32,160 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:36235-0x10180a7ca410000 connected 2023-07-20 02:55:32,187 DEBUG [Listener at localhost/35915] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 02:55:32,188 DEBUG [Listener at localhost/35915] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 02:55:32,192 DEBUG [Listener at localhost/35915] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-20 02:55:32,200 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36235 2023-07-20 02:55:32,201 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36235 2023-07-20 02:55:32,201 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36235 2023-07-20 02:55:32,202 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36235 2023-07-20 02:55:32,202 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36235 2023-07-20 02:55:32,208 INFO [Listener at localhost/35915] master.HMaster(444): hbase.rootdir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61, hbase.cluster.distributed=false 2023-07-20 02:55:32,286 INFO [Listener at localhost/35915] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-20 02:55:32,287 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:32,287 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:32,287 INFO [Listener at localhost/35915] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:55:32,287 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:32,288 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:55:32,293 INFO [Listener at localhost/35915] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:55:32,296 INFO [Listener at localhost/35915] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39331 2023-07-20 02:55:32,299 INFO [Listener at localhost/35915] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 02:55:32,305 DEBUG [Listener at localhost/35915] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 02:55:32,307 INFO [Listener at localhost/35915] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:32,309 INFO [Listener at localhost/35915] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:32,311 INFO [Listener at localhost/35915] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39331 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:32,316 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:393310x0, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:32,316 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:39331-0x10180a7ca410001 connected 2023-07-20 02:55:32,317 DEBUG [Listener at localhost/35915] zookeeper.ZKUtil(164): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 02:55:32,318 DEBUG [Listener at localhost/35915] zookeeper.ZKUtil(164): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 02:55:32,319 DEBUG [Listener at localhost/35915] zookeeper.ZKUtil(164): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-20 02:55:32,319 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39331 2023-07-20 02:55:32,320 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39331 2023-07-20 02:55:32,320 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39331 2023-07-20 02:55:32,320 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39331 2023-07-20 02:55:32,321 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39331 2023-07-20 02:55:32,333 INFO [Listener at localhost/35915] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-20 02:55:32,333 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:32,334 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:32,334 INFO [Listener at localhost/35915] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:55:32,334 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:32,334 INFO [Listener at localhost/35915] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:55:32,335 INFO [Listener at localhost/35915] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:55:32,336 INFO [Listener at localhost/35915] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46185 2023-07-20 02:55:32,337 INFO [Listener at localhost/35915] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 02:55:32,338 DEBUG [Listener at localhost/35915] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 02:55:32,339 INFO [Listener at localhost/35915] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:32,340 INFO [Listener at localhost/35915] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:32,342 INFO [Listener at localhost/35915] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46185 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:32,346 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:461850x0, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:32,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46185-0x10180a7ca410002 connected 2023-07-20 02:55:32,347 DEBUG [Listener at localhost/35915] zookeeper.ZKUtil(164): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 02:55:32,348 DEBUG [Listener at localhost/35915] zookeeper.ZKUtil(164): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 02:55:32,349 DEBUG [Listener at localhost/35915] zookeeper.ZKUtil(164): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-20 02:55:32,350 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46185 2023-07-20 02:55:32,350 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46185 2023-07-20 02:55:32,350 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46185 2023-07-20 02:55:32,351 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46185 2023-07-20 02:55:32,351 DEBUG [Listener at localhost/35915] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46185 2023-07-20 02:55:32,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,36235,1689821731026 2023-07-20 02:55:32,361 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-20 02:55:32,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,36235,1689821731026 2023-07-20 02:55:32,382 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-20 02:55:32,383 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-20 02:55:32,383 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-20 02:55:32,383 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:32,384 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-20 02:55:32,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,36235,1689821731026 from backup master directory 2023-07-20 02:55:32,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-20 02:55:32,389 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,36235,1689821731026 2023-07-20 02:55:32,389 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-20 02:55:32,390 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:55:32,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,36235,1689821731026 2023-07-20 02:55:32,393 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-20 02:55:32,394 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-20 02:55:32,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/hbase.id with ID: bbd9fa59-2213-4359-9606-c616f1cd40eb 2023-07-20 02:55:32,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:32,539 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:32,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3e74e971 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:32,624 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@40f84985, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:32,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:32,652 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-20 02:55:32,673 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-20 02:55:32,673 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-20 02:55:32,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-20 02:55:32,681 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-20 02:55:32,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:32,722 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store-tmp 2023-07-20 02:55:32,763 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:32,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 02:55:32,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:32,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:32,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 02:55:32,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:32,765 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:32,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:55:32,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36235,1689821731026 2023-07-20 02:55:32,793 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36235%2C1689821731026, suffix=, logDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36235,1689821731026, archiveDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/oldWALs, maxLogs=10 2023-07-20 02:55:32,866 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK] 2023-07-20 02:55:32,866 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK] 2023-07-20 02:55:32,875 DEBUG [RS-EventLoopGroup-4-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-20 02:55:32,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36235,1689821731026/jenkins-hbase4.apache.org%2C36235%2C1689821731026.1689821732808 2023-07-20 02:55:32,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK], DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK]] 2023-07-20 02:55:32,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:32,950 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:32,953 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:32,954 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:33,012 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:33,019 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-20 02:55:33,049 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-20 02:55:33,062 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:33,067 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:33,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:33,089 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:33,094 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 02:55:33,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65736932, jitterRate=-0.020443379878997803}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 02:55:33,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:55:33,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-20 02:55:33,122 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-20 02:55:33,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-20 02:55:33,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-20 02:55:33,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-20 02:55:33,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 33 msec 2023-07-20 02:55:33,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-20 02:55:33,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-20 02:55:33,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-20 02:55:33,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-20 02:55:33,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-20 02:55:33,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-20 02:55:33,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-20 02:55:33,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-20 02:55:33,237 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:33,239 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-20 02:55:33,239 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-20 02:55:33,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-20 02:55:33,259 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-20 02:55:33,259 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-20 02:55:33,259 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:33,260 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-20 02:55:33,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,36235,1689821731026, sessionid=0x10180a7ca410000, setting cluster-up flag (Was=false) 2023-07-20 02:55:33,279 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:33,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-20 02:55:33,288 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36235,1689821731026 2023-07-20 02:55:33,294 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:33,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-20 02:55:33,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36235,1689821731026 2023-07-20 02:55:33,304 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.hbase-snapshot/.tmp 2023-07-20 02:55:33,355 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(951): ClusterId : bbd9fa59-2213-4359-9606-c616f1cd40eb 2023-07-20 02:55:33,355 INFO [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(951): ClusterId : bbd9fa59-2213-4359-9606-c616f1cd40eb 2023-07-20 02:55:33,360 DEBUG [RS:0;jenkins-hbase4:39331] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 02:55:33,360 DEBUG [RS:1;jenkins-hbase4:46185] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 02:55:33,368 DEBUG [RS:1;jenkins-hbase4:46185] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 02:55:33,368 DEBUG [RS:0;jenkins-hbase4:39331] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 02:55:33,368 DEBUG [RS:1;jenkins-hbase4:46185] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 02:55:33,368 DEBUG [RS:0;jenkins-hbase4:39331] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 02:55:33,371 DEBUG [RS:1;jenkins-hbase4:46185] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 02:55:33,371 DEBUG [RS:0;jenkins-hbase4:39331] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 02:55:33,373 DEBUG [RS:1;jenkins-hbase4:46185] zookeeper.ReadOnlyZKClient(139): Connect 0x6ee0915f to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:33,373 DEBUG [RS:0;jenkins-hbase4:39331] zookeeper.ReadOnlyZKClient(139): Connect 0x16f27e27 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:33,380 DEBUG [RS:1;jenkins-hbase4:46185] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@602762f3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:33,381 DEBUG [RS:0;jenkins-hbase4:39331] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18358289, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:33,381 DEBUG [RS:1;jenkins-hbase4:46185] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6418fa2f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:55:33,381 DEBUG [RS:0;jenkins-hbase4:39331] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c0a1557, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:55:33,408 DEBUG [RS:1;jenkins-hbase4:46185] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:46185 2023-07-20 02:55:33,411 DEBUG [RS:0;jenkins-hbase4:39331] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:39331 2023-07-20 02:55:33,414 INFO [RS:0;jenkins-hbase4:39331] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 02:55:33,414 INFO [RS:1;jenkins-hbase4:46185] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 02:55:33,415 INFO [RS:1;jenkins-hbase4:46185] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 02:55:33,415 INFO [RS:0;jenkins-hbase4:39331] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 02:55:33,415 DEBUG [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 02:55:33,415 DEBUG [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 02:55:33,418 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36235,1689821731026 with isa=jenkins-hbase4.apache.org/172.31.14.131:46185, startcode=1689821732333 2023-07-20 02:55:33,418 INFO [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36235,1689821731026 with isa=jenkins-hbase4.apache.org/172.31.14.131:39331, startcode=1689821732285 2023-07-20 02:55:33,428 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-20 02:55:33,438 DEBUG [RS:1;jenkins-hbase4:46185] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 02:55:33,438 DEBUG [RS:0;jenkins-hbase4:39331] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 02:55:33,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:33,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:33,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:33,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:33,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-20 02:55:33,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:55:33,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689821763453 2023-07-20 02:55:33,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-20 02:55:33,457 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-20 02:55:33,458 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-20 02:55:33,467 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:33,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-20 02:55:33,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-20 02:55:33,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-20 02:55:33,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-20 02:55:33,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-20 02:55:33,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-20 02:55:33,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-20 02:55:33,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-20 02:55:33,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-20 02:55:33,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-20 02:55:33,513 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821733512,5,FailOnTimeoutGroup] 2023-07-20 02:55:33,513 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54837, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 02:55:33,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821733513,5,FailOnTimeoutGroup] 2023-07-20 02:55:33,513 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59593, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 02:55:33,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-20 02:55:33,518 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,518 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,543 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-20 02:55:33,545 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-20 02:55:33,545 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61 2023-07-20 02:55:33,546 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36235] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:33,564 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36235] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:33,566 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:33,569 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 02:55:33,571 DEBUG [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61 2023-07-20 02:55:33,571 DEBUG [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61 2023-07-20 02:55:33,572 DEBUG [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37763 2023-07-20 02:55:33,572 DEBUG [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37763 2023-07-20 02:55:33,572 DEBUG [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 02:55:33,572 DEBUG [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 02:55:33,572 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/info 2023-07-20 02:55:33,573 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 02:55:33,574 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:33,574 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 02:55:33,577 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:55:33,578 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 02:55:33,578 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 02:55:33,579 DEBUG [RS:0;jenkins-hbase4:39331] zookeeper.ZKUtil(162): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:33,579 WARN [RS:0;jenkins-hbase4:39331] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:55:33,579 INFO [RS:0;jenkins-hbase4:39331] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:33,581 DEBUG [RS:1;jenkins-hbase4:46185] zookeeper.ZKUtil(162): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:33,581 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:33,581 WARN [RS:1;jenkins-hbase4:46185] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:55:33,581 DEBUG [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:33,581 INFO [RS:1;jenkins-hbase4:46185] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:33,581 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 02:55:33,582 DEBUG [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:33,582 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39331,1689821732285] 2023-07-20 02:55:33,582 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46185,1689821732333] 2023-07-20 02:55:33,587 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/table 2023-07-20 02:55:33,588 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 02:55:33,588 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:33,590 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740 2023-07-20 02:55:33,591 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740 2023-07-20 02:55:33,594 DEBUG [RS:1;jenkins-hbase4:46185] zookeeper.ZKUtil(162): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:33,594 DEBUG [RS:0;jenkins-hbase4:39331] zookeeper.ZKUtil(162): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:33,594 DEBUG [RS:1;jenkins-hbase4:46185] zookeeper.ZKUtil(162): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:33,594 DEBUG [RS:0;jenkins-hbase4:39331] zookeeper.ZKUtil(162): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:33,596 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 02:55:33,598 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 02:55:33,603 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 02:55:33,603 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74006140, jitterRate=0.10277742147445679}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 02:55:33,603 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 02:55:33,604 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 02:55:33,604 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 02:55:33,604 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 02:55:33,604 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 02:55:33,604 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 02:55:33,604 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 02:55:33,605 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 02:55:33,607 DEBUG [RS:0;jenkins-hbase4:39331] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:55:33,607 DEBUG [RS:1;jenkins-hbase4:46185] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:55:33,610 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-20 02:55:33,611 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-20 02:55:33,617 INFO [RS:0;jenkins-hbase4:39331] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 02:55:33,617 INFO [RS:1;jenkins-hbase4:46185] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 02:55:33,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-20 02:55:33,634 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-20 02:55:33,637 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-20 02:55:33,640 INFO [RS:1;jenkins-hbase4:46185] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 02:55:33,640 INFO [RS:0;jenkins-hbase4:39331] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 02:55:33,644 INFO [RS:1;jenkins-hbase4:46185] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 02:55:33,644 INFO [RS:0;jenkins-hbase4:39331] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 02:55:33,644 INFO [RS:1;jenkins-hbase4:46185] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,644 INFO [RS:0;jenkins-hbase4:39331] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,645 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 02:55:33,646 INFO [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 02:55:33,653 INFO [RS:0;jenkins-hbase4:39331] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,653 INFO [RS:1;jenkins-hbase4:46185] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,653 DEBUG [RS:0;jenkins-hbase4:39331] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,653 DEBUG [RS:1;jenkins-hbase4:46185] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,654 DEBUG [RS:0;jenkins-hbase4:39331] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,654 DEBUG [RS:1;jenkins-hbase4:46185] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,654 DEBUG [RS:0;jenkins-hbase4:39331] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,654 DEBUG [RS:1;jenkins-hbase4:46185] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,654 DEBUG [RS:0;jenkins-hbase4:39331] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,654 DEBUG [RS:1;jenkins-hbase4:46185] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,654 DEBUG [RS:0;jenkins-hbase4:39331] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,654 DEBUG [RS:1;jenkins-hbase4:46185] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,654 DEBUG [RS:0;jenkins-hbase4:39331] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:55:33,654 DEBUG [RS:1;jenkins-hbase4:46185] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:55:33,654 DEBUG [RS:0;jenkins-hbase4:39331] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,654 DEBUG [RS:1;jenkins-hbase4:46185] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,655 DEBUG [RS:0;jenkins-hbase4:39331] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,655 DEBUG [RS:1;jenkins-hbase4:46185] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,655 DEBUG [RS:0;jenkins-hbase4:39331] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,655 DEBUG [RS:1;jenkins-hbase4:46185] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,655 DEBUG [RS:0;jenkins-hbase4:39331] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,655 DEBUG [RS:1;jenkins-hbase4:46185] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:33,656 INFO [RS:0;jenkins-hbase4:39331] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,657 INFO [RS:1;jenkins-hbase4:46185] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,657 INFO [RS:0;jenkins-hbase4:39331] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,657 INFO [RS:1;jenkins-hbase4:46185] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,657 INFO [RS:0;jenkins-hbase4:39331] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,657 INFO [RS:1;jenkins-hbase4:46185] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,675 INFO [RS:0;jenkins-hbase4:39331] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 02:55:33,675 INFO [RS:1;jenkins-hbase4:46185] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 02:55:33,679 INFO [RS:1;jenkins-hbase4:46185] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46185,1689821732333-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,679 INFO [RS:0;jenkins-hbase4:39331] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39331,1689821732285-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:33,700 INFO [RS:1;jenkins-hbase4:46185] regionserver.Replication(203): jenkins-hbase4.apache.org,46185,1689821732333 started 2023-07-20 02:55:33,700 INFO [RS:0;jenkins-hbase4:39331] regionserver.Replication(203): jenkins-hbase4.apache.org,39331,1689821732285 started 2023-07-20 02:55:33,700 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46185,1689821732333, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46185, sessionid=0x10180a7ca410002 2023-07-20 02:55:33,700 INFO [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,39331,1689821732285, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39331, sessionid=0x10180a7ca410001 2023-07-20 02:55:33,701 DEBUG [RS:0;jenkins-hbase4:39331] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 02:55:33,701 DEBUG [RS:1;jenkins-hbase4:46185] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 02:55:33,701 DEBUG [RS:0;jenkins-hbase4:39331] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:33,701 DEBUG [RS:1;jenkins-hbase4:46185] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:33,701 DEBUG [RS:0;jenkins-hbase4:39331] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39331,1689821732285' 2023-07-20 02:55:33,701 DEBUG [RS:1;jenkins-hbase4:46185] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46185,1689821732333' 2023-07-20 02:55:33,702 DEBUG [RS:1;jenkins-hbase4:46185] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-20 02:55:33,702 DEBUG [RS:0;jenkins-hbase4:39331] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-20 02:55:33,702 DEBUG [RS:0;jenkins-hbase4:39331] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-20 02:55:33,702 DEBUG [RS:1;jenkins-hbase4:46185] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-20 02:55:33,703 DEBUG [RS:1;jenkins-hbase4:46185] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 02:55:33,703 DEBUG [RS:0;jenkins-hbase4:39331] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 02:55:33,703 DEBUG [RS:1;jenkins-hbase4:46185] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 02:55:33,703 DEBUG [RS:0;jenkins-hbase4:39331] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 02:55:33,703 DEBUG [RS:1;jenkins-hbase4:46185] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:33,703 DEBUG [RS:0;jenkins-hbase4:39331] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:33,704 DEBUG [RS:0;jenkins-hbase4:39331] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39331,1689821732285' 2023-07-20 02:55:33,704 DEBUG [RS:0;jenkins-hbase4:39331] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-20 02:55:33,704 DEBUG [RS:1;jenkins-hbase4:46185] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46185,1689821732333' 2023-07-20 02:55:33,704 DEBUG [RS:1;jenkins-hbase4:46185] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-20 02:55:33,704 DEBUG [RS:0;jenkins-hbase4:39331] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-20 02:55:33,704 DEBUG [RS:1;jenkins-hbase4:46185] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-20 02:55:33,705 DEBUG [RS:0;jenkins-hbase4:39331] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 02:55:33,705 DEBUG [RS:1;jenkins-hbase4:46185] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 02:55:33,705 INFO [RS:0;jenkins-hbase4:39331] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 02:55:33,705 INFO [RS:1;jenkins-hbase4:46185] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 02:55:33,705 INFO [RS:0;jenkins-hbase4:39331] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 02:55:33,705 INFO [RS:1;jenkins-hbase4:46185] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 02:55:33,789 DEBUG [jenkins-hbase4:36235] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-07-20 02:55:33,794 DEBUG [jenkins-hbase4:36235] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 02:55:33,800 DEBUG [jenkins-hbase4:36235] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 02:55:33,800 DEBUG [jenkins-hbase4:36235] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 02:55:33,800 DEBUG [jenkins-hbase4:36235] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 02:55:33,803 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46185,1689821732333, state=OPENING 2023-07-20 02:55:33,815 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-20 02:55:33,816 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:33,817 INFO [RS:0;jenkins-hbase4:39331] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39331%2C1689821732285, suffix=, logDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285, archiveDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs, maxLogs=10 2023-07-20 02:55:33,817 INFO [RS:1;jenkins-hbase4:46185] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46185%2C1689821732333, suffix=, logDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333, archiveDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs, maxLogs=10 2023-07-20 02:55:33,818 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-20 02:55:33,822 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46185,1689821732333}] 2023-07-20 02:55:33,846 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK] 2023-07-20 02:55:33,846 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK] 2023-07-20 02:55:33,851 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK] 2023-07-20 02:55:33,856 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK] 2023-07-20 02:55:33,867 INFO [RS:0;jenkins-hbase4:39331] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822 2023-07-20 02:55:33,867 INFO [RS:1;jenkins-hbase4:46185] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822 2023-07-20 02:55:33,868 DEBUG [RS:0;jenkins-hbase4:39331] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK], DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK]] 2023-07-20 02:55:33,868 DEBUG [RS:1;jenkins-hbase4:46185] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK], DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK]] 2023-07-20 02:55:34,014 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:34,017 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:34,020 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43212, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:55:34,032 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-20 02:55:34,033 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:34,037 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46185%2C1689821732333.meta, suffix=.meta, logDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333, archiveDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs, maxLogs=10 2023-07-20 02:55:34,056 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK] 2023-07-20 02:55:34,056 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK] 2023-07-20 02:55:34,069 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.meta.1689821734039.meta 2023-07-20 02:55:34,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK], DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK]] 2023-07-20 02:55:34,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:34,072 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-20 02:55:34,091 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-20 02:55:34,130 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-20 02:55:34,136 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-20 02:55:34,136 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:34,136 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-20 02:55:34,136 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-20 02:55:34,139 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 02:55:34,141 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/info 2023-07-20 02:55:34,141 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/info 2023-07-20 02:55:34,142 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 02:55:34,143 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:34,143 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 02:55:34,144 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:55:34,145 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:55:34,145 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 02:55:34,146 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:34,146 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 02:55:34,148 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/table 2023-07-20 02:55:34,148 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/table 2023-07-20 02:55:34,148 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 02:55:34,149 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:34,151 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740 2023-07-20 02:55:34,154 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740 2023-07-20 02:55:34,157 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 02:55:34,160 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 02:55:34,162 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69262399, jitterRate=0.03209017217159271}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 02:55:34,162 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 02:55:34,176 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689821734007 2023-07-20 02:55:34,194 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-20 02:55:34,195 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46185,1689821732333, state=OPEN 2023-07-20 02:55:34,197 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-20 02:55:34,200 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-20 02:55:34,200 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-20 02:55:34,207 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-20 02:55:34,207 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46185,1689821732333 in 379 msec 2023-07-20 02:55:34,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-20 02:55:34,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 589 msec 2023-07-20 02:55:34,224 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 872 msec 2023-07-20 02:55:34,224 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689821734224, completionTime=-1 2023-07-20 02:55:34,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-07-20 02:55:34,225 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-20 02:55:34,289 DEBUG [hconnection-0x6aef63d2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:34,292 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43226, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:34,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-07-20 02:55:34,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689821794311 2023-07-20 02:55:34,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689821854312 2023-07-20 02:55:34,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 86 msec 2023-07-20 02:55:34,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36235,1689821731026-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:34,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36235,1689821731026-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:34,345 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36235,1689821731026-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:34,346 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:36235, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:34,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:34,355 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-20 02:55:34,363 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-20 02:55:34,364 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:34,375 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-20 02:55:34,379 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 02:55:34,382 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 02:55:34,415 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:55:34,419 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d empty. 2023-07-20 02:55:34,420 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:55:34,420 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-20 02:55:34,456 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-20 02:55:34,459 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1af5feb4e30a65bed26cf0457e90777d, NAME => 'hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp 2023-07-20 02:55:34,483 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:34,484 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 1af5feb4e30a65bed26cf0457e90777d, disabling compactions & flushes 2023-07-20 02:55:34,484 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:34,484 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:34,484 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. after waiting 0 ms 2023-07-20 02:55:34,484 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:34,484 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:34,484 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 1af5feb4e30a65bed26cf0457e90777d: 2023-07-20 02:55:34,489 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 02:55:34,506 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821734492"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689821734492"}]},"ts":"1689821734492"} 2023-07-20 02:55:34,539 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 02:55:34,541 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 02:55:34,546 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821734542"}]},"ts":"1689821734542"} 2023-07-20 02:55:34,550 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-20 02:55:34,554 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 02:55:34,557 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 02:55:34,557 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 02:55:34,557 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 02:55:34,559 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1af5feb4e30a65bed26cf0457e90777d, ASSIGN}] 2023-07-20 02:55:34,562 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1af5feb4e30a65bed26cf0457e90777d, ASSIGN 2023-07-20 02:55:34,564 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=1af5feb4e30a65bed26cf0457e90777d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46185,1689821732333; forceNewPlan=false, retain=false 2023-07-20 02:55:34,716 INFO [jenkins-hbase4:36235] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 02:55:34,718 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=1af5feb4e30a65bed26cf0457e90777d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:34,719 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821734718"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821734718"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821734718"}]},"ts":"1689821734718"} 2023-07-20 02:55:34,727 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 1af5feb4e30a65bed26cf0457e90777d, server=jenkins-hbase4.apache.org,46185,1689821732333}] 2023-07-20 02:55:34,889 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:34,890 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1af5feb4e30a65bed26cf0457e90777d, NAME => 'hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:34,891 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:55:34,891 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:34,891 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:55:34,891 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:55:34,894 INFO [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:55:34,897 DEBUG [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/info 2023-07-20 02:55:34,897 DEBUG [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/info 2023-07-20 02:55:34,898 INFO [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1af5feb4e30a65bed26cf0457e90777d columnFamilyName info 2023-07-20 02:55:34,899 INFO [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] regionserver.HStore(310): Store=1af5feb4e30a65bed26cf0457e90777d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:34,901 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:55:34,902 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:55:34,907 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:55:34,924 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 02:55:34,926 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1af5feb4e30a65bed26cf0457e90777d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63158316, jitterRate=-0.05886775255203247}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 02:55:34,926 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1af5feb4e30a65bed26cf0457e90777d: 2023-07-20 02:55:34,930 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d., pid=6, masterSystemTime=1689821734882 2023-07-20 02:55:34,936 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:34,936 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:34,937 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=1af5feb4e30a65bed26cf0457e90777d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:34,938 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821734937"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821734937"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821734937"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821734937"}]},"ts":"1689821734937"} 2023-07-20 02:55:34,950 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-20 02:55:34,951 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 1af5feb4e30a65bed26cf0457e90777d, server=jenkins-hbase4.apache.org,46185,1689821732333 in 218 msec 2023-07-20 02:55:34,956 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-20 02:55:34,958 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 02:55:34,959 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=1af5feb4e30a65bed26cf0457e90777d, ASSIGN in 391 msec 2023-07-20 02:55:34,960 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821734959"}]},"ts":"1689821734959"} 2023-07-20 02:55:34,968 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-20 02:55:34,973 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 02:55:34,977 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 608 msec 2023-07-20 02:55:34,979 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-07-20 02:55:34,981 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-07-20 02:55:34,981 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:35,025 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-20 02:55:35,052 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-20 02:55:35,061 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 46 msec 2023-07-20 02:55:35,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-20 02:55:35,085 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-20 02:55:35,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 23 msec 2023-07-20 02:55:35,107 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-20 02:55:35,112 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-20 02:55:35,113 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.722sec 2023-07-20 02:55:35,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-20 02:55:35,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-20 02:55:35,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-20 02:55:35,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36235,1689821731026-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-20 02:55:35,122 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36235,1689821731026-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-20 02:55:35,133 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-20 02:55:35,163 DEBUG [Listener at localhost/35915] zookeeper.ReadOnlyZKClient(139): Connect 0x7f32269b to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:35,174 DEBUG [Listener at localhost/35915] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@448b0fe6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:35,198 DEBUG [hconnection-0x340b1bee-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:35,221 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43234, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:35,233 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,36235,1689821731026 2023-07-20 02:55:35,233 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-20 02:55:35,233 INFO [Listener at localhost/35915] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/cluster_316081d3-021c-b0fe-c72f-4dd54f1e7285, deleteOnExit=true 2023-07-20 02:55:35,233 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-20 02:55:35,248 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/test.cache.data in system properties and HBase conf 2023-07-20 02:55:35,248 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/hadoop.tmp.dir in system properties and HBase conf 2023-07-20 02:55:35,248 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/hadoop.log.dir in system properties and HBase conf 2023-07-20 02:55:35,249 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-20 02:55:35,249 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-20 02:55:35,249 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-20 02:55:35,249 DEBUG [Listener at localhost/35915] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-20 02:55:35,250 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-20 02:55:35,250 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-20 02:55:35,250 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-20 02:55:35,250 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-20 02:55:35,251 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-20 02:55:35,251 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-20 02:55:35,251 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-20 02:55:35,251 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-20 02:55:35,251 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-20 02:55:35,251 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/nfs.dump.dir in system properties and HBase conf 2023-07-20 02:55:35,251 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/java.io.tmpdir in system properties and HBase conf 2023-07-20 02:55:35,252 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-20 02:55:35,252 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-20 02:55:35,252 INFO [Listener at localhost/35915] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-20 02:55:35,269 WARN [Listener at localhost/35915] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-20 02:55:35,269 WARN [Listener at localhost/35915] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-20 02:55:35,332 WARN [Listener at localhost/35915] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 02:55:35,335 INFO [Listener at localhost/35915] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 02:55:35,347 INFO [Listener at localhost/35915] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/java.io.tmpdir/Jetty_localhost_39421_hdfs____.fm47pl/webapp 2023-07-20 02:55:35,448 INFO [Listener at localhost/35915] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39421 2023-07-20 02:55:35,459 WARN [Listener at localhost/35915] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-20 02:55:35,459 WARN [Listener at localhost/35915] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-20 02:55:35,567 WARN [Listener at localhost/40517] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 02:55:35,582 WARN [Listener at localhost/40517] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 02:55:35,584 WARN [Listener at localhost/40517] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 02:55:35,585 INFO [Listener at localhost/40517] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 02:55:35,590 INFO [Listener at localhost/40517] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/java.io.tmpdir/Jetty_localhost_36655_datanode____nmhim2/webapp 2023-07-20 02:55:35,687 INFO [Listener at localhost/40517] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36655 2023-07-20 02:55:35,696 WARN [Listener at localhost/37533] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 02:55:35,715 WARN [Listener at localhost/37533] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 02:55:35,718 WARN [Listener at localhost/37533] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 02:55:35,719 INFO [Listener at localhost/37533] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 02:55:35,723 INFO [Listener at localhost/37533] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/java.io.tmpdir/Jetty_localhost_32839_datanode____xjkuh2/webapp 2023-07-20 02:55:35,814 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x97e02a30de32ed1: Processing first storage report for DS-87be6ac9-bf85-4b5b-85aa-76790df4118c from datanode f7d2b3c4-486a-49ed-81f3-c03d9100d412 2023-07-20 02:55:35,814 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x97e02a30de32ed1: from storage DS-87be6ac9-bf85-4b5b-85aa-76790df4118c node DatanodeRegistration(127.0.0.1:34619, datanodeUuid=f7d2b3c4-486a-49ed-81f3-c03d9100d412, infoPort=35175, infoSecurePort=0, ipcPort=37533, storageInfo=lv=-57;cid=testClusterID;nsid=1179189412;c=1689821735277), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 02:55:35,814 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x97e02a30de32ed1: Processing first storage report for DS-7005dd4f-4af6-4f08-b64f-e1d78f37a691 from datanode f7d2b3c4-486a-49ed-81f3-c03d9100d412 2023-07-20 02:55:35,815 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x97e02a30de32ed1: from storage DS-7005dd4f-4af6-4f08-b64f-e1d78f37a691 node DatanodeRegistration(127.0.0.1:34619, datanodeUuid=f7d2b3c4-486a-49ed-81f3-c03d9100d412, infoPort=35175, infoSecurePort=0, ipcPort=37533, storageInfo=lv=-57;cid=testClusterID;nsid=1179189412;c=1689821735277), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 02:55:35,874 INFO [Listener at localhost/37533] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32839 2023-07-20 02:55:35,912 WARN [Listener at localhost/33773] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 02:55:35,996 WARN [Listener at localhost/33773] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 02:55:35,999 WARN [Listener at localhost/33773] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 02:55:36,000 INFO [Listener at localhost/33773] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 02:55:36,009 INFO [Listener at localhost/33773] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/java.io.tmpdir/Jetty_localhost_34339_datanode____y23kxr/webapp 2023-07-20 02:55:36,119 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x813200fe22c8ee99: Processing first storage report for DS-b74a8495-a7ff-448d-bd64-a7d9320b197d from datanode f924fba1-da90-407e-a2a9-bca2b317681e 2023-07-20 02:55:36,119 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x813200fe22c8ee99: from storage DS-b74a8495-a7ff-448d-bd64-a7d9320b197d node DatanodeRegistration(127.0.0.1:38071, datanodeUuid=f924fba1-da90-407e-a2a9-bca2b317681e, infoPort=44883, infoSecurePort=0, ipcPort=33773, storageInfo=lv=-57;cid=testClusterID;nsid=1179189412;c=1689821735277), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 02:55:36,119 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x813200fe22c8ee99: Processing first storage report for DS-8eabcaf7-bf93-4d1e-a185-2aace15cbe22 from datanode f924fba1-da90-407e-a2a9-bca2b317681e 2023-07-20 02:55:36,119 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x813200fe22c8ee99: from storage DS-8eabcaf7-bf93-4d1e-a185-2aace15cbe22 node DatanodeRegistration(127.0.0.1:38071, datanodeUuid=f924fba1-da90-407e-a2a9-bca2b317681e, infoPort=44883, infoSecurePort=0, ipcPort=33773, storageInfo=lv=-57;cid=testClusterID;nsid=1179189412;c=1689821735277), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 02:55:36,141 INFO [Listener at localhost/33773] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34339 2023-07-20 02:55:36,158 WARN [Listener at localhost/41727] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 02:55:36,207 WARN [Listener at localhost/41727] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 02:55:36,215 WARN [Listener at localhost/41727] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 02:55:36,216 INFO [Listener at localhost/41727] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 02:55:36,232 INFO [Listener at localhost/41727] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/java.io.tmpdir/Jetty_localhost_38931_datanode____.oe93k7/webapp 2023-07-20 02:55:36,307 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x41613870fc69d265: Processing first storage report for DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb from datanode 6d82a86a-3163-4833-bb75-a6eb1299bdd9 2023-07-20 02:55:36,307 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x41613870fc69d265: from storage DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb node DatanodeRegistration(127.0.0.1:34649, datanodeUuid=6d82a86a-3163-4833-bb75-a6eb1299bdd9, infoPort=33949, infoSecurePort=0, ipcPort=41727, storageInfo=lv=-57;cid=testClusterID;nsid=1179189412;c=1689821735277), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 02:55:36,307 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x41613870fc69d265: Processing first storage report for DS-170c010c-0dd0-4177-8fe2-ba9c95d407b7 from datanode 6d82a86a-3163-4833-bb75-a6eb1299bdd9 2023-07-20 02:55:36,307 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x41613870fc69d265: from storage DS-170c010c-0dd0-4177-8fe2-ba9c95d407b7 node DatanodeRegistration(127.0.0.1:34649, datanodeUuid=6d82a86a-3163-4833-bb75-a6eb1299bdd9, infoPort=33949, infoSecurePort=0, ipcPort=41727, storageInfo=lv=-57;cid=testClusterID;nsid=1179189412;c=1689821735277), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 02:55:36,352 INFO [Listener at localhost/41727] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38931 2023-07-20 02:55:36,366 WARN [Listener at localhost/46633] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 02:55:36,483 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x31f9042e6f47d5a2: Processing first storage report for DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a from datanode d05519fe-bfcd-4db3-937b-29de0dad3891 2023-07-20 02:55:36,483 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x31f9042e6f47d5a2: from storage DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a node DatanodeRegistration(127.0.0.1:42001, datanodeUuid=d05519fe-bfcd-4db3-937b-29de0dad3891, infoPort=36863, infoSecurePort=0, ipcPort=46633, storageInfo=lv=-57;cid=testClusterID;nsid=1179189412;c=1689821735277), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 02:55:36,483 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x31f9042e6f47d5a2: Processing first storage report for DS-a90ea1c1-d8e3-4f77-8dfe-329d7bb633c5 from datanode d05519fe-bfcd-4db3-937b-29de0dad3891 2023-07-20 02:55:36,484 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x31f9042e6f47d5a2: from storage DS-a90ea1c1-d8e3-4f77-8dfe-329d7bb633c5 node DatanodeRegistration(127.0.0.1:42001, datanodeUuid=d05519fe-bfcd-4db3-937b-29de0dad3891, infoPort=36863, infoSecurePort=0, ipcPort=46633, storageInfo=lv=-57;cid=testClusterID;nsid=1179189412;c=1689821735277), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 02:55:36,584 DEBUG [Listener at localhost/46633] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42 2023-07-20 02:55:36,584 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,585 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,617 INFO [Listener at localhost/46633] util.FSUtils(471): Created version file at hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16 with version=8 2023-07-20 02:55:36,618 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/hbase-staging 2023-07-20 02:55:36,620 INFO [Listener at localhost/46633] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 02:55:36,620 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,620 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,620 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:55:36,620 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,620 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:55:36,620 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:55:36,622 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45829 2023-07-20 02:55:36,622 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,624 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,625 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45829 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:36,628 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:458290x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:36,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:45829-0x10180a7ca410007 connected 2023-07-20 02:55:36,643 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:55:36,643 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:36,644 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 02:55:36,644 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45829 2023-07-20 02:55:36,644 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45829 2023-07-20 02:55:36,645 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45829 2023-07-20 02:55:36,645 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45829 2023-07-20 02:55:36,645 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45829 2023-07-20 02:55:36,646 INFO [Listener at localhost/46633] master.HMaster(444): hbase.rootdir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16, hbase.cluster.distributed=false 2023-07-20 02:55:36,661 INFO [Listener at localhost/46633] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 02:55:36,661 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,662 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,662 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:55:36,662 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,662 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:55:36,662 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:55:36,664 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44133 2023-07-20 02:55:36,664 INFO [Listener at localhost/46633] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 02:55:36,666 DEBUG [Listener at localhost/46633] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 02:55:36,667 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,668 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,670 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44133 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:36,673 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:441330x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:36,675 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:44133-0x10180a7ca410008 connected 2023-07-20 02:55:36,675 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:55:36,675 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:36,676 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 02:55:36,676 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44133 2023-07-20 02:55:36,678 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44133 2023-07-20 02:55:36,679 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44133 2023-07-20 02:55:36,679 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44133 2023-07-20 02:55:36,680 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44133 2023-07-20 02:55:36,692 INFO [Listener at localhost/46633] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 02:55:36,692 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,692 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,692 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:55:36,692 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,692 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:55:36,692 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:55:36,694 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40959 2023-07-20 02:55:36,695 INFO [Listener at localhost/46633] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 02:55:36,696 DEBUG [Listener at localhost/46633] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 02:55:36,697 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,698 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,699 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40959 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:36,702 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:409590x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:36,704 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40959-0x10180a7ca410009 connected 2023-07-20 02:55:36,704 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:55:36,704 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:36,705 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 02:55:36,705 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40959 2023-07-20 02:55:36,705 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40959 2023-07-20 02:55:36,706 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40959 2023-07-20 02:55:36,706 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40959 2023-07-20 02:55:36,706 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40959 2023-07-20 02:55:36,718 INFO [Listener at localhost/46633] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 02:55:36,718 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,719 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,719 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:55:36,719 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,719 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:55:36,719 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:55:36,724 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38819 2023-07-20 02:55:36,724 INFO [Listener at localhost/46633] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 02:55:36,725 DEBUG [Listener at localhost/46633] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 02:55:36,726 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,727 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,728 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38819 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:36,731 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:388190x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:36,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38819-0x10180a7ca41000a connected 2023-07-20 02:55:36,733 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:55:36,733 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:36,734 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 02:55:36,734 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38819 2023-07-20 02:55:36,736 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38819 2023-07-20 02:55:36,736 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38819 2023-07-20 02:55:36,736 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38819 2023-07-20 02:55:36,737 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38819 2023-07-20 02:55:36,748 INFO [Listener at localhost/46633] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 02:55:36,748 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,748 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,748 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:55:36,749 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:36,749 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:55:36,749 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:55:36,751 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46489 2023-07-20 02:55:36,751 INFO [Listener at localhost/46633] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 02:55:36,752 DEBUG [Listener at localhost/46633] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 02:55:36,753 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,754 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,755 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46489 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:36,760 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:464890x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:36,761 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46489-0x10180a7ca41000b connected 2023-07-20 02:55:36,762 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:55:36,762 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:36,763 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 02:55:36,763 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46489 2023-07-20 02:55:36,763 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46489 2023-07-20 02:55:36,763 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46489 2023-07-20 02:55:36,764 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46489 2023-07-20 02:55:36,764 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46489 2023-07-20 02:55:36,765 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,45829,1689821736619 2023-07-20 02:55:36,767 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 02:55:36,768 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,45829,1689821736619 2023-07-20 02:55:36,769 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 02:55:36,769 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 02:55:36,770 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 02:55:36,769 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 02:55:36,769 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 02:55:36,770 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:36,772 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 02:55:36,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 02:55:36,774 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,45829,1689821736619 from backup master directory 2023-07-20 02:55:36,776 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,45829,1689821736619 2023-07-20 02:55:36,776 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 02:55:36,776 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:55:36,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,45829,1689821736619 2023-07-20 02:55:36,805 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/hbase.id with ID: 69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:55:36,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:36,826 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:36,847 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x21722d70 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:36,853 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7aadc12, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:36,854 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:36,854 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-20 02:55:36,855 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:36,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store-tmp 2023-07-20 02:55:36,880 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:36,880 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 02:55:36,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:36,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:36,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 02:55:36,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:36,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:36,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:55:36,882 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,45829,1689821736619 2023-07-20 02:55:36,886 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45829%2C1689821736619, suffix=, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,45829,1689821736619, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/oldWALs, maxLogs=10 2023-07-20 02:55:36,905 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK] 2023-07-20 02:55:36,907 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK] 2023-07-20 02:55:36,908 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK] 2023-07-20 02:55:36,911 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,45829,1689821736619/jenkins-hbase4.apache.org%2C45829%2C1689821736619.1689821736886 2023-07-20 02:55:36,911 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK], DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK], DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK]] 2023-07-20 02:55:36,911 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:36,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:36,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:36,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:36,915 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:36,917 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-20 02:55:36,918 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-20 02:55:36,918 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:36,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:36,921 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:36,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:36,929 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 02:55:36,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71919451, jitterRate=0.07168333232402802}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 02:55:36,929 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:55:36,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-20 02:55:36,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-20 02:55:36,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-20 02:55:36,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-20 02:55:36,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-20 02:55:36,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 1 msec 2023-07-20 02:55:36,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-20 02:55:36,935 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-20 02:55:36,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-20 02:55:36,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-20 02:55:36,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-20 02:55:36,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-20 02:55:36,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-20 02:55:36,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-20 02:55:36,950 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:36,952 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-20 02:55:36,952 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-20 02:55:36,953 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-20 02:55:36,954 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 02:55:36,955 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 02:55:36,955 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 02:55:36,955 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 02:55:36,955 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 02:55:36,955 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:36,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,45829,1689821736619, sessionid=0x10180a7ca410007, setting cluster-up flag (Was=false) 2023-07-20 02:55:36,961 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:36,970 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-20 02:55:36,972 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45829,1689821736619 2023-07-20 02:55:36,975 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:36,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-20 02:55:36,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45829,1689821736619 2023-07-20 02:55:36,985 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.hbase-snapshot/.tmp 2023-07-20 02:55:36,988 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-20 02:55:36,988 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:36,988 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:36,988 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:36,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:36,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-20 02:55:36,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:36,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:55:36,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:36,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689821766991 2023-07-20 02:55:36,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-20 02:55:36,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-20 02:55:36,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-20 02:55:36,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-20 02:55:36,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-20 02:55:36,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-20 02:55:36,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:36,992 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-20 02:55:36,993 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-20 02:55:36,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-20 02:55:36,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-20 02:55:36,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-20 02:55:36,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-20 02:55:36,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-20 02:55:36,995 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:36,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821736995,5,FailOnTimeoutGroup] 2023-07-20 02:55:36,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821736996,5,FailOnTimeoutGroup] 2023-07-20 02:55:36,996 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:36,996 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-20 02:55:36,996 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:36,996 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,023 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-20 02:55:37,024 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-20 02:55:37,024 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16 2023-07-20 02:55:37,041 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:37,043 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 02:55:37,045 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info 2023-07-20 02:55:37,046 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 02:55:37,046 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:37,046 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 02:55:37,048 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:55:37,048 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 02:55:37,049 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:37,049 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 02:55:37,051 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table 2023-07-20 02:55:37,051 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 02:55:37,052 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:37,053 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740 2023-07-20 02:55:37,054 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740 2023-07-20 02:55:37,057 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 02:55:37,058 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 02:55:37,063 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 02:55:37,064 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70814568, jitterRate=0.055219292640686035}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 02:55:37,064 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 02:55:37,064 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 02:55:37,064 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 02:55:37,064 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 02:55:37,064 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 02:55:37,064 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 02:55:37,064 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 02:55:37,065 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 02:55:37,067 INFO [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(951): ClusterId : 69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:55:37,067 INFO [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(951): ClusterId : 69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:55:37,067 INFO [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(951): ClusterId : 69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:55:37,067 INFO [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(951): ClusterId : 69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:55:37,068 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-20 02:55:37,068 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-20 02:55:37,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-20 02:55:37,070 DEBUG [RS:0;jenkins-hbase4:44133] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 02:55:37,070 DEBUG [RS:2;jenkins-hbase4:38819] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 02:55:37,070 DEBUG [RS:3;jenkins-hbase4:46489] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 02:55:37,070 DEBUG [RS:1;jenkins-hbase4:40959] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 02:55:37,073 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-20 02:55:37,077 DEBUG [RS:0;jenkins-hbase4:44133] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 02:55:37,077 DEBUG [RS:0;jenkins-hbase4:44133] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 02:55:37,078 DEBUG [RS:2;jenkins-hbase4:38819] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 02:55:37,078 DEBUG [RS:2;jenkins-hbase4:38819] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 02:55:37,078 DEBUG [RS:3;jenkins-hbase4:46489] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 02:55:37,080 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-20 02:55:37,080 DEBUG [RS:3;jenkins-hbase4:46489] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 02:55:37,082 DEBUG [RS:1;jenkins-hbase4:40959] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 02:55:37,082 DEBUG [RS:1;jenkins-hbase4:40959] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 02:55:37,083 DEBUG [RS:0;jenkins-hbase4:44133] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 02:55:37,084 DEBUG [RS:0;jenkins-hbase4:44133] zookeeper.ReadOnlyZKClient(139): Connect 0x2032c975 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:37,092 DEBUG [RS:1;jenkins-hbase4:40959] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 02:55:37,092 DEBUG [RS:3;jenkins-hbase4:46489] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 02:55:37,094 DEBUG [RS:1;jenkins-hbase4:40959] zookeeper.ReadOnlyZKClient(139): Connect 0x3e3197a8 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:37,095 DEBUG [RS:2;jenkins-hbase4:38819] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 02:55:37,095 DEBUG [RS:3;jenkins-hbase4:46489] zookeeper.ReadOnlyZKClient(139): Connect 0x54ccb3c5 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:37,097 DEBUG [RS:2;jenkins-hbase4:38819] zookeeper.ReadOnlyZKClient(139): Connect 0x0c28d054 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:37,098 DEBUG [RS:0;jenkins-hbase4:44133] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ef52303, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:37,098 DEBUG [RS:0;jenkins-hbase4:44133] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2acaa205, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:55:37,106 DEBUG [RS:3;jenkins-hbase4:46489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@50f33d84, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:37,107 DEBUG [RS:2;jenkins-hbase4:38819] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6d0bb24, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:37,107 DEBUG [RS:3;jenkins-hbase4:46489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@15cdf142, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:55:37,107 DEBUG [RS:2;jenkins-hbase4:38819] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@43d2359a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:55:37,111 DEBUG [RS:1;jenkins-hbase4:40959] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@30f67e1e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:37,111 DEBUG [RS:1;jenkins-hbase4:40959] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@34c9fcd8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:55:37,114 DEBUG [RS:0;jenkins-hbase4:44133] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:44133 2023-07-20 02:55:37,114 INFO [RS:0;jenkins-hbase4:44133] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 02:55:37,114 INFO [RS:0;jenkins-hbase4:44133] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 02:55:37,114 DEBUG [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 02:55:37,115 INFO [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45829,1689821736619 with isa=jenkins-hbase4.apache.org/172.31.14.131:44133, startcode=1689821736661 2023-07-20 02:55:37,115 DEBUG [RS:0;jenkins-hbase4:44133] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 02:55:37,120 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:46489 2023-07-20 02:55:37,120 INFO [RS:3;jenkins-hbase4:46489] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 02:55:37,120 INFO [RS:3;jenkins-hbase4:46489] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 02:55:37,120 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 02:55:37,121 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45507, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 02:55:37,122 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45829] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:37,122 INFO [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45829,1689821736619 with isa=jenkins-hbase4.apache.org/172.31.14.131:46489, startcode=1689821736748 2023-07-20 02:55:37,122 DEBUG [RS:3;jenkins-hbase4:46489] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 02:55:37,123 DEBUG [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16 2023-07-20 02:55:37,123 DEBUG [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40517 2023-07-20 02:55:37,123 DEBUG [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 02:55:37,124 DEBUG [RS:2;jenkins-hbase4:38819] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:38819 2023-07-20 02:55:37,124 INFO [RS:2;jenkins-hbase4:38819] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 02:55:37,124 INFO [RS:2;jenkins-hbase4:38819] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 02:55:37,124 DEBUG [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 02:55:37,125 DEBUG [RS:1;jenkins-hbase4:40959] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:40959 2023-07-20 02:55:37,125 INFO [RS:1;jenkins-hbase4:40959] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 02:55:37,125 INFO [RS:1;jenkins-hbase4:40959] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 02:55:37,125 DEBUG [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 02:55:37,127 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:37,127 INFO [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45829,1689821736619 with isa=jenkins-hbase4.apache.org/172.31.14.131:40959, startcode=1689821736691 2023-07-20 02:55:37,127 INFO [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45829,1689821736619 with isa=jenkins-hbase4.apache.org/172.31.14.131:38819, startcode=1689821736718 2023-07-20 02:55:37,128 DEBUG [RS:1;jenkins-hbase4:40959] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 02:55:37,128 DEBUG [RS:2;jenkins-hbase4:38819] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 02:55:37,128 DEBUG [RS:0;jenkins-hbase4:44133] zookeeper.ZKUtil(162): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:37,128 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43011, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 02:55:37,128 WARN [RS:0;jenkins-hbase4:44133] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:55:37,128 INFO [RS:0;jenkins-hbase4:44133] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:37,128 DEBUG [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:37,128 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45829] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:37,130 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16 2023-07-20 02:55:37,130 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40517 2023-07-20 02:55:37,130 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 02:55:37,130 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39249, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 02:55:37,131 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45829] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:37,131 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,44133,1689821736661] 2023-07-20 02:55:37,132 DEBUG [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16 2023-07-20 02:55:37,132 DEBUG [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40517 2023-07-20 02:55:37,132 DEBUG [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 02:55:37,132 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:37,133 DEBUG [RS:3;jenkins-hbase4:46489] zookeeper.ZKUtil(162): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:37,133 WARN [RS:3;jenkins-hbase4:46489] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:55:37,133 INFO [RS:3;jenkins-hbase4:46489] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:37,133 DEBUG [RS:2;jenkins-hbase4:38819] zookeeper.ZKUtil(162): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:37,133 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:37,134 WARN [RS:2;jenkins-hbase4:38819] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:55:37,134 INFO [RS:2;jenkins-hbase4:38819] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:37,134 DEBUG [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:37,135 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38039, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 02:55:37,136 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45829] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:37,137 DEBUG [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16 2023-07-20 02:55:37,137 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46489,1689821736748] 2023-07-20 02:55:37,137 DEBUG [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40517 2023-07-20 02:55:37,137 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38819,1689821736718] 2023-07-20 02:55:37,137 DEBUG [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 02:55:37,140 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:37,140 DEBUG [RS:1;jenkins-hbase4:40959] zookeeper.ZKUtil(162): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:37,141 WARN [RS:1;jenkins-hbase4:40959] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:55:37,141 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40959,1689821736691] 2023-07-20 02:55:37,141 INFO [RS:1;jenkins-hbase4:40959] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:37,142 DEBUG [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:37,144 DEBUG [RS:3;jenkins-hbase4:46489] zookeeper.ZKUtil(162): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:37,144 DEBUG [RS:3;jenkins-hbase4:46489] zookeeper.ZKUtil(162): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:37,145 DEBUG [RS:3;jenkins-hbase4:46489] zookeeper.ZKUtil(162): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:37,146 DEBUG [RS:3;jenkins-hbase4:46489] zookeeper.ZKUtil(162): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:37,150 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:55:37,153 INFO [RS:3;jenkins-hbase4:46489] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 02:55:37,156 DEBUG [RS:2;jenkins-hbase4:38819] zookeeper.ZKUtil(162): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:37,156 DEBUG [RS:0;jenkins-hbase4:44133] zookeeper.ZKUtil(162): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:37,156 DEBUG [RS:2;jenkins-hbase4:38819] zookeeper.ZKUtil(162): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:37,157 DEBUG [RS:0;jenkins-hbase4:44133] zookeeper.ZKUtil(162): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:37,157 DEBUG [RS:2;jenkins-hbase4:38819] zookeeper.ZKUtil(162): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:37,157 DEBUG [RS:0;jenkins-hbase4:44133] zookeeper.ZKUtil(162): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:37,158 DEBUG [RS:2;jenkins-hbase4:38819] zookeeper.ZKUtil(162): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:37,158 DEBUG [RS:0;jenkins-hbase4:44133] zookeeper.ZKUtil(162): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:37,159 DEBUG [RS:1;jenkins-hbase4:40959] zookeeper.ZKUtil(162): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:37,159 DEBUG [RS:0;jenkins-hbase4:44133] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:55:37,159 DEBUG [RS:1;jenkins-hbase4:40959] zookeeper.ZKUtil(162): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:37,160 INFO [RS:0;jenkins-hbase4:44133] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 02:55:37,160 DEBUG [RS:1;jenkins-hbase4:40959] zookeeper.ZKUtil(162): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:37,161 DEBUG [RS:1;jenkins-hbase4:40959] zookeeper.ZKUtil(162): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:37,162 DEBUG [RS:1;jenkins-hbase4:40959] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:55:37,163 INFO [RS:3;jenkins-hbase4:46489] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 02:55:37,163 DEBUG [RS:2;jenkins-hbase4:38819] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:55:37,163 INFO [RS:1;jenkins-hbase4:40959] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 02:55:37,163 INFO [RS:2;jenkins-hbase4:38819] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 02:55:37,163 INFO [RS:0;jenkins-hbase4:44133] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 02:55:37,167 INFO [RS:0;jenkins-hbase4:44133] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 02:55:37,167 INFO [RS:3;jenkins-hbase4:46489] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 02:55:37,167 INFO [RS:0;jenkins-hbase4:44133] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,167 INFO [RS:3;jenkins-hbase4:46489] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,167 INFO [RS:1;jenkins-hbase4:40959] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 02:55:37,170 INFO [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 02:55:37,171 INFO [RS:2;jenkins-hbase4:38819] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 02:55:37,173 INFO [RS:1;jenkins-hbase4:40959] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 02:55:37,173 INFO [RS:1;jenkins-hbase4:40959] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,173 INFO [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 02:55:37,173 INFO [RS:2;jenkins-hbase4:38819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 02:55:37,173 INFO [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 02:55:37,173 INFO [RS:2;jenkins-hbase4:38819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,174 INFO [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 02:55:37,176 INFO [RS:1;jenkins-hbase4:40959] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,176 DEBUG [RS:1;jenkins-hbase4:40959] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,176 DEBUG [RS:1;jenkins-hbase4:40959] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,176 DEBUG [RS:1;jenkins-hbase4:40959] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,176 DEBUG [RS:1;jenkins-hbase4:40959] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,177 DEBUG [RS:1;jenkins-hbase4:40959] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,177 DEBUG [RS:1;jenkins-hbase4:40959] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:55:37,177 DEBUG [RS:1;jenkins-hbase4:40959] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,177 DEBUG [RS:1;jenkins-hbase4:40959] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,177 DEBUG [RS:1;jenkins-hbase4:40959] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,177 INFO [RS:2;jenkins-hbase4:38819] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,177 DEBUG [RS:1;jenkins-hbase4:40959] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,177 DEBUG [RS:2;jenkins-hbase4:38819] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,177 INFO [RS:0;jenkins-hbase4:44133] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,177 DEBUG [RS:2;jenkins-hbase4:38819] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,178 DEBUG [RS:2;jenkins-hbase4:38819] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,178 DEBUG [RS:2;jenkins-hbase4:38819] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,178 DEBUG [RS:2;jenkins-hbase4:38819] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,178 DEBUG [RS:2;jenkins-hbase4:38819] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:55:37,178 DEBUG [RS:2;jenkins-hbase4:38819] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,178 DEBUG [RS:2;jenkins-hbase4:38819] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,178 DEBUG [RS:2;jenkins-hbase4:38819] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,178 DEBUG [RS:2;jenkins-hbase4:38819] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,179 DEBUG [RS:0;jenkins-hbase4:44133] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,179 DEBUG [RS:0;jenkins-hbase4:44133] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,179 DEBUG [RS:0;jenkins-hbase4:44133] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,179 DEBUG [RS:0;jenkins-hbase4:44133] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,179 DEBUG [RS:0;jenkins-hbase4:44133] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,179 DEBUG [RS:0;jenkins-hbase4:44133] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:55:37,179 DEBUG [RS:0;jenkins-hbase4:44133] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,179 DEBUG [RS:0;jenkins-hbase4:44133] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,179 DEBUG [RS:0;jenkins-hbase4:44133] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,179 DEBUG [RS:0;jenkins-hbase4:44133] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,182 INFO [RS:2;jenkins-hbase4:38819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,183 INFO [RS:2;jenkins-hbase4:38819] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,183 INFO [RS:2;jenkins-hbase4:38819] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,186 INFO [RS:1;jenkins-hbase4:40959] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,187 INFO [RS:3;jenkins-hbase4:46489] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,187 INFO [RS:1;jenkins-hbase4:40959] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,187 INFO [RS:1;jenkins-hbase4:40959] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,190 INFO [RS:0;jenkins-hbase4:44133] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,190 INFO [RS:0;jenkins-hbase4:44133] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,190 INFO [RS:0;jenkins-hbase4:44133] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,190 DEBUG [RS:3;jenkins-hbase4:46489] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,191 DEBUG [RS:3;jenkins-hbase4:46489] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,191 DEBUG [RS:3;jenkins-hbase4:46489] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,191 DEBUG [RS:3;jenkins-hbase4:46489] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,191 DEBUG [RS:3;jenkins-hbase4:46489] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,191 DEBUG [RS:3;jenkins-hbase4:46489] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:55:37,191 DEBUG [RS:3;jenkins-hbase4:46489] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,191 DEBUG [RS:3;jenkins-hbase4:46489] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,191 DEBUG [RS:3;jenkins-hbase4:46489] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,191 DEBUG [RS:3;jenkins-hbase4:46489] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:37,192 INFO [RS:3;jenkins-hbase4:46489] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,193 INFO [RS:3;jenkins-hbase4:46489] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,193 INFO [RS:3;jenkins-hbase4:46489] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,202 INFO [RS:1;jenkins-hbase4:40959] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 02:55:37,202 INFO [RS:2;jenkins-hbase4:38819] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 02:55:37,202 INFO [RS:1;jenkins-hbase4:40959] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40959,1689821736691-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,202 INFO [RS:2;jenkins-hbase4:38819] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38819,1689821736718-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,204 INFO [RS:3;jenkins-hbase4:46489] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 02:55:37,204 INFO [RS:3;jenkins-hbase4:46489] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46489,1689821736748-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,207 INFO [RS:0;jenkins-hbase4:44133] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 02:55:37,207 INFO [RS:0;jenkins-hbase4:44133] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44133,1689821736661-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,213 INFO [RS:1;jenkins-hbase4:40959] regionserver.Replication(203): jenkins-hbase4.apache.org,40959,1689821736691 started 2023-07-20 02:55:37,213 INFO [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,40959,1689821736691, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40959, sessionid=0x10180a7ca410009 2023-07-20 02:55:37,213 DEBUG [RS:1;jenkins-hbase4:40959] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 02:55:37,213 DEBUG [RS:1;jenkins-hbase4:40959] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:37,213 DEBUG [RS:1;jenkins-hbase4:40959] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40959,1689821736691' 2023-07-20 02:55:37,213 DEBUG [RS:1;jenkins-hbase4:40959] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 02:55:37,214 DEBUG [RS:1;jenkins-hbase4:40959] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 02:55:37,215 DEBUG [RS:1;jenkins-hbase4:40959] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 02:55:37,215 DEBUG [RS:1;jenkins-hbase4:40959] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 02:55:37,215 DEBUG [RS:1;jenkins-hbase4:40959] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:37,215 DEBUG [RS:1;jenkins-hbase4:40959] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40959,1689821736691' 2023-07-20 02:55:37,215 DEBUG [RS:1;jenkins-hbase4:40959] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 02:55:37,215 INFO [RS:3;jenkins-hbase4:46489] regionserver.Replication(203): jenkins-hbase4.apache.org,46489,1689821736748 started 2023-07-20 02:55:37,216 INFO [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46489,1689821736748, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46489, sessionid=0x10180a7ca41000b 2023-07-20 02:55:37,216 DEBUG [RS:3;jenkins-hbase4:46489] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 02:55:37,216 DEBUG [RS:3;jenkins-hbase4:46489] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:37,216 DEBUG [RS:3;jenkins-hbase4:46489] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46489,1689821736748' 2023-07-20 02:55:37,216 DEBUG [RS:3;jenkins-hbase4:46489] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 02:55:37,216 DEBUG [RS:1;jenkins-hbase4:40959] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 02:55:37,216 DEBUG [RS:3;jenkins-hbase4:46489] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 02:55:37,216 DEBUG [RS:1;jenkins-hbase4:40959] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 02:55:37,216 INFO [RS:1;jenkins-hbase4:40959] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 02:55:37,216 INFO [RS:1;jenkins-hbase4:40959] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 02:55:37,217 DEBUG [RS:3;jenkins-hbase4:46489] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 02:55:37,217 DEBUG [RS:3;jenkins-hbase4:46489] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 02:55:37,217 DEBUG [RS:3;jenkins-hbase4:46489] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:37,217 DEBUG [RS:3;jenkins-hbase4:46489] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46489,1689821736748' 2023-07-20 02:55:37,217 DEBUG [RS:3;jenkins-hbase4:46489] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 02:55:37,217 DEBUG [RS:3;jenkins-hbase4:46489] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 02:55:37,217 DEBUG [RS:3;jenkins-hbase4:46489] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 02:55:37,217 INFO [RS:3;jenkins-hbase4:46489] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 02:55:37,217 INFO [RS:3;jenkins-hbase4:46489] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 02:55:37,219 INFO [RS:2;jenkins-hbase4:38819] regionserver.Replication(203): jenkins-hbase4.apache.org,38819,1689821736718 started 2023-07-20 02:55:37,219 INFO [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38819,1689821736718, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38819, sessionid=0x10180a7ca41000a 2023-07-20 02:55:37,219 DEBUG [RS:2;jenkins-hbase4:38819] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 02:55:37,219 DEBUG [RS:2;jenkins-hbase4:38819] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:37,219 DEBUG [RS:2;jenkins-hbase4:38819] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38819,1689821736718' 2023-07-20 02:55:37,220 DEBUG [RS:2;jenkins-hbase4:38819] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 02:55:37,220 DEBUG [RS:2;jenkins-hbase4:38819] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 02:55:37,220 DEBUG [RS:2;jenkins-hbase4:38819] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 02:55:37,220 DEBUG [RS:2;jenkins-hbase4:38819] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 02:55:37,220 DEBUG [RS:2;jenkins-hbase4:38819] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:37,221 DEBUG [RS:2;jenkins-hbase4:38819] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38819,1689821736718' 2023-07-20 02:55:37,221 DEBUG [RS:2;jenkins-hbase4:38819] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 02:55:37,221 DEBUG [RS:2;jenkins-hbase4:38819] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 02:55:37,221 DEBUG [RS:2;jenkins-hbase4:38819] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 02:55:37,221 INFO [RS:2;jenkins-hbase4:38819] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 02:55:37,221 INFO [RS:2;jenkins-hbase4:38819] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 02:55:37,222 INFO [RS:0;jenkins-hbase4:44133] regionserver.Replication(203): jenkins-hbase4.apache.org,44133,1689821736661 started 2023-07-20 02:55:37,222 INFO [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,44133,1689821736661, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:44133, sessionid=0x10180a7ca410008 2023-07-20 02:55:37,223 DEBUG [RS:0;jenkins-hbase4:44133] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 02:55:37,223 DEBUG [RS:0;jenkins-hbase4:44133] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:37,223 DEBUG [RS:0;jenkins-hbase4:44133] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44133,1689821736661' 2023-07-20 02:55:37,223 DEBUG [RS:0;jenkins-hbase4:44133] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 02:55:37,223 DEBUG [RS:0;jenkins-hbase4:44133] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 02:55:37,223 DEBUG [RS:0;jenkins-hbase4:44133] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 02:55:37,223 DEBUG [RS:0;jenkins-hbase4:44133] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 02:55:37,223 DEBUG [RS:0;jenkins-hbase4:44133] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:37,223 DEBUG [RS:0;jenkins-hbase4:44133] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44133,1689821736661' 2023-07-20 02:55:37,223 DEBUG [RS:0;jenkins-hbase4:44133] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 02:55:37,224 DEBUG [RS:0;jenkins-hbase4:44133] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 02:55:37,225 DEBUG [RS:0;jenkins-hbase4:44133] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 02:55:37,225 INFO [RS:0;jenkins-hbase4:44133] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 02:55:37,225 INFO [RS:0;jenkins-hbase4:44133] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 02:55:37,231 DEBUG [jenkins-hbase4:45829] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-07-20 02:55:37,231 DEBUG [jenkins-hbase4:45829] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 02:55:37,234 DEBUG [jenkins-hbase4:45829] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 02:55:37,234 DEBUG [jenkins-hbase4:45829] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 02:55:37,234 DEBUG [jenkins-hbase4:45829] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-20 02:55:37,234 DEBUG [jenkins-hbase4:45829] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-20 02:55:37,234 DEBUG [jenkins-hbase4:45829] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 02:55:37,235 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46489,1689821736748, state=OPENING 2023-07-20 02:55:37,237 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-20 02:55:37,238 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:37,239 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 02:55:37,239 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46489,1689821736748}] 2023-07-20 02:55:37,320 INFO [RS:1;jenkins-hbase4:40959] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40959%2C1689821736691, suffix=, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,40959,1689821736691, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs, maxLogs=10 2023-07-20 02:55:37,321 INFO [RS:3;jenkins-hbase4:46489] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46489%2C1689821736748, suffix=, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,46489,1689821736748, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs, maxLogs=10 2023-07-20 02:55:37,326 INFO [RS:2;jenkins-hbase4:38819] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38819%2C1689821736718, suffix=, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38819,1689821736718, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs, maxLogs=10 2023-07-20 02:55:37,336 INFO [RS:0;jenkins-hbase4:44133] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44133%2C1689821736661, suffix=, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,44133,1689821736661, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs, maxLogs=10 2023-07-20 02:55:37,362 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK] 2023-07-20 02:55:37,362 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK] 2023-07-20 02:55:37,393 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK] 2023-07-20 02:55:37,394 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK] 2023-07-20 02:55:37,394 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK] 2023-07-20 02:55:37,398 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK] 2023-07-20 02:55:37,441 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:37,441 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:37,448 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK] 2023-07-20 02:55:37,451 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK] 2023-07-20 02:55:37,451 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK] 2023-07-20 02:55:37,452 INFO [RS:1;jenkins-hbase4:40959] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,40959,1689821736691/jenkins-hbase4.apache.org%2C40959%2C1689821736691.1689821737321 2023-07-20 02:55:37,462 DEBUG [RS:1;jenkins-hbase4:40959] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK], DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK], DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK]] 2023-07-20 02:55:37,477 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK] 2023-07-20 02:55:37,477 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK] 2023-07-20 02:55:37,478 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45046, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:55:37,488 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK] 2023-07-20 02:55:37,488 INFO [RS:2;jenkins-hbase4:38819] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38819,1689821736718/jenkins-hbase4.apache.org%2C38819%2C1689821736718.1689821737328 2023-07-20 02:55:37,492 INFO [RS:3;jenkins-hbase4:46489] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,46489,1689821736748/jenkins-hbase4.apache.org%2C46489%2C1689821736748.1689821737322 2023-07-20 02:55:37,500 DEBUG [RS:2;jenkins-hbase4:38819] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK], DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK], DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK]] 2023-07-20 02:55:37,502 DEBUG [RS:3;jenkins-hbase4:46489] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK], DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK], DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK]] 2023-07-20 02:55:37,502 INFO [RS:0;jenkins-hbase4:44133] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,44133,1689821736661/jenkins-hbase4.apache.org%2C44133%2C1689821736661.1689821737342 2023-07-20 02:55:37,506 DEBUG [RS:0;jenkins-hbase4:44133] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK], DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK], DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK]] 2023-07-20 02:55:37,517 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-20 02:55:37,517 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:37,521 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46489%2C1689821736748.meta, suffix=.meta, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,46489,1689821736748, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs, maxLogs=10 2023-07-20 02:55:37,547 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK] 2023-07-20 02:55:37,551 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK] 2023-07-20 02:55:37,551 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK] 2023-07-20 02:55:37,560 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,46489,1689821736748/jenkins-hbase4.apache.org%2C46489%2C1689821736748.meta.1689821737522.meta 2023-07-20 02:55:37,560 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK], DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK], DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK]] 2023-07-20 02:55:37,561 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:37,561 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-20 02:55:37,561 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-20 02:55:37,562 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-20 02:55:37,562 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-20 02:55:37,562 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:37,562 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-20 02:55:37,562 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-20 02:55:37,565 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 02:55:37,567 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info 2023-07-20 02:55:37,567 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info 2023-07-20 02:55:37,567 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 02:55:37,568 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:37,568 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 02:55:37,570 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:55:37,570 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:55:37,570 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 02:55:37,571 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:37,571 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 02:55:37,572 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table 2023-07-20 02:55:37,573 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table 2023-07-20 02:55:37,573 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 02:55:37,574 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:37,576 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740 2023-07-20 02:55:37,580 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740 2023-07-20 02:55:37,583 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 02:55:37,585 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 02:55:37,586 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59711680, jitterRate=-0.11022663116455078}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 02:55:37,586 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 02:55:37,591 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689821737435 2023-07-20 02:55:37,597 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-20 02:55:37,598 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-20 02:55:37,599 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46489,1689821736748, state=OPEN 2023-07-20 02:55:37,601 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-20 02:55:37,601 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 02:55:37,604 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-20 02:55:37,604 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46489,1689821736748 in 362 msec 2023-07-20 02:55:37,608 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-20 02:55:37,608 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 536 msec 2023-07-20 02:55:37,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 623 msec 2023-07-20 02:55:37,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689821737611, completionTime=-1 2023-07-20 02:55:37,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-07-20 02:55:37,612 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-20 02:55:37,615 DEBUG [hconnection-0x51c1cde5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:37,617 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45054, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:37,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-07-20 02:55:37,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689821797619 2023-07-20 02:55:37,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689821857619 2023-07-20 02:55:37,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 7 msec 2023-07-20 02:55:37,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45829,1689821736619-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45829,1689821736619-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45829,1689821736619-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:45829, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:37,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-20 02:55:37,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:37,631 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-20 02:55:37,631 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-20 02:55:37,635 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 02:55:37,636 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 02:55:37,641 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:37,642 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7 empty. 2023-07-20 02:55:37,643 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:37,643 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-20 02:55:37,697 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-20 02:55:37,698 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 956fe6fd22cc055b7d5c09ea7e2fd4a7, NAME => 'hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp 2023-07-20 02:55:37,724 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:37,724 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 956fe6fd22cc055b7d5c09ea7e2fd4a7, disabling compactions & flushes 2023-07-20 02:55:37,724 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:37,724 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:37,724 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. after waiting 0 ms 2023-07-20 02:55:37,724 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:37,724 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:37,724 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 956fe6fd22cc055b7d5c09ea7e2fd4a7: 2023-07-20 02:55:37,729 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 02:55:37,733 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821737733"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689821737733"}]},"ts":"1689821737733"} 2023-07-20 02:55:37,737 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 02:55:37,739 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 02:55:37,739 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821737739"}]},"ts":"1689821737739"} 2023-07-20 02:55:37,741 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-20 02:55:37,746 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 02:55:37,747 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 02:55:37,747 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 02:55:37,747 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-20 02:55:37,747 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-20 02:55:37,747 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 02:55:37,748 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN}] 2023-07-20 02:55:37,751 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN 2023-07-20 02:55:37,753 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44133,1689821736661; forceNewPlan=false, retain=false 2023-07-20 02:55:37,903 INFO [jenkins-hbase4:45829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 02:55:37,905 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=956fe6fd22cc055b7d5c09ea7e2fd4a7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:37,905 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821737904"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821737904"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821737904"}]},"ts":"1689821737904"} 2023-07-20 02:55:37,908 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 956fe6fd22cc055b7d5c09ea7e2fd4a7, server=jenkins-hbase4.apache.org,44133,1689821736661}] 2023-07-20 02:55:38,062 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:38,062 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:38,064 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44674, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:55:38,072 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:38,072 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 956fe6fd22cc055b7d5c09ea7e2fd4a7, NAME => 'hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:38,072 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:38,073 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:38,073 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:38,073 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:38,076 INFO [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:38,079 DEBUG [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/info 2023-07-20 02:55:38,079 DEBUG [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/info 2023-07-20 02:55:38,080 INFO [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 956fe6fd22cc055b7d5c09ea7e2fd4a7 columnFamilyName info 2023-07-20 02:55:38,081 INFO [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] regionserver.HStore(310): Store=956fe6fd22cc055b7d5c09ea7e2fd4a7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:38,083 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:38,085 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:38,117 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:38,124 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 02:55:38,125 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 956fe6fd22cc055b7d5c09ea7e2fd4a7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63479625, jitterRate=-0.054079875349998474}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 02:55:38,125 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 956fe6fd22cc055b7d5c09ea7e2fd4a7: 2023-07-20 02:55:38,127 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7., pid=6, masterSystemTime=1689821738062 2023-07-20 02:55:38,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:38,134 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:38,135 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=956fe6fd22cc055b7d5c09ea7e2fd4a7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:38,136 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821738135"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821738135"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821738135"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821738135"}]},"ts":"1689821738135"} 2023-07-20 02:55:38,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-20 02:55:38,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 956fe6fd22cc055b7d5c09ea7e2fd4a7, server=jenkins-hbase4.apache.org,44133,1689821736661 in 231 msec 2023-07-20 02:55:38,147 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-20 02:55:38,149 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN in 396 msec 2023-07-20 02:55:38,151 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 02:55:38,151 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821738151"}]},"ts":"1689821738151"} 2023-07-20 02:55:38,153 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-20 02:55:38,157 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 02:55:38,160 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 528 msec 2023-07-20 02:55:38,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-07-20 02:55:38,237 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-07-20 02:55:38,237 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:38,241 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:38,244 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44682, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:38,248 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-20 02:55:38,266 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-20 02:55:38,273 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 23 msec 2023-07-20 02:55:38,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-20 02:55:38,296 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-20 02:55:38,301 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 19 msec 2023-07-20 02:55:38,306 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-20 02:55:38,309 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-20 02:55:38,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.533sec 2023-07-20 02:55:38,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-20 02:55:38,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-20 02:55:38,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-20 02:55:38,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45829,1689821736619-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-20 02:55:38,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45829,1689821736619-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-20 02:55:38,315 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-20 02:55:38,368 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(139): Connect 0x607cb094 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:38,375 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17a1e8e0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:38,376 DEBUG [hconnection-0x409f3945-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:38,378 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45062, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:38,381 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,45829,1689821736619 2023-07-20 02:55:38,394 DEBUG [Listener at localhost/46633] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-20 02:55:38,396 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38218, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-20 02:55:38,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:38,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-20 02:55:38,416 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 02:55:38,419 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 02:55:38,422 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:38,422 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-20 02:55:38,423 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a empty. 2023-07-20 02:55:38,424 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:38,424 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-20 02:55:38,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 02:55:38,460 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-20 02:55:38,462 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 300903cec0fd231ca88313bcf649662a, NAME => 't1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp 2023-07-20 02:55:38,479 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:38,479 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 300903cec0fd231ca88313bcf649662a, disabling compactions & flushes 2023-07-20 02:55:38,479 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:38,479 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:38,479 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. after waiting 0 ms 2023-07-20 02:55:38,479 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:38,479 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:38,479 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 300903cec0fd231ca88313bcf649662a: 2023-07-20 02:55:38,484 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 02:55:38,486 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821738485"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689821738485"}]},"ts":"1689821738485"} 2023-07-20 02:55:38,488 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 02:55:38,490 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 02:55:38,490 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821738490"}]},"ts":"1689821738490"} 2023-07-20 02:55:38,492 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-20 02:55:38,497 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 02:55:38,499 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 02:55:38,499 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 02:55:38,499 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 02:55:38,499 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=300903cec0fd231ca88313bcf649662a, ASSIGN}] 2023-07-20 02:55:38,502 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=300903cec0fd231ca88313bcf649662a, ASSIGN 2023-07-20 02:55:38,503 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=300903cec0fd231ca88313bcf649662a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39331,1689821732285; forceNewPlan=false, retain=false 2023-07-20 02:55:38,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 02:55:38,653 INFO [jenkins-hbase4:36235] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 02:55:38,655 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=300903cec0fd231ca88313bcf649662a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:38,656 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821738655"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821738655"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821738655"}]},"ts":"1689821738655"} 2023-07-20 02:55:38,658 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 300903cec0fd231ca88313bcf649662a, server=jenkins-hbase4.apache.org,39331,1689821732285}] 2023-07-20 02:55:38,700 INFO [jenkins-hbase4:46185Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:55:38,700 INFO [jenkins-hbase4:39331Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:55:38,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 02:55:38,824 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:38,825 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:38,830 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38580, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:55:38,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:38,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 300903cec0fd231ca88313bcf649662a, NAME => 't1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:38,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:38,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:38,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:38,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:38,841 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:38,844 DEBUG [StoreOpener-300903cec0fd231ca88313bcf649662a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/cf1 2023-07-20 02:55:38,844 DEBUG [StoreOpener-300903cec0fd231ca88313bcf649662a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/cf1 2023-07-20 02:55:38,845 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 300903cec0fd231ca88313bcf649662a columnFamilyName cf1 2023-07-20 02:55:38,845 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] regionserver.HStore(310): Store=300903cec0fd231ca88313bcf649662a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:38,846 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:38,848 DEBUG [StoreOpener-300903cec0fd231ca88313bcf649662a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/norep 2023-07-20 02:55:38,848 DEBUG [StoreOpener-300903cec0fd231ca88313bcf649662a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/norep 2023-07-20 02:55:38,849 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 300903cec0fd231ca88313bcf649662a columnFamilyName norep 2023-07-20 02:55:38,849 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] regionserver.HStore(310): Store=300903cec0fd231ca88313bcf649662a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:38,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:38,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:38,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 02:55:38,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:38,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 02:55:38,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 300903cec0fd231ca88313bcf649662a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65865806, jitterRate=-0.01852300763130188}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 02:55:38,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 300903cec0fd231ca88313bcf649662a: 2023-07-20 02:55:38,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a., pid=11, masterSystemTime=1689821738824 2023-07-20 02:55:38,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:38,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:38,873 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=300903cec0fd231ca88313bcf649662a, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:38,874 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821738873"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821738873"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821738873"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821738873"}]},"ts":"1689821738873"} 2023-07-20 02:55:38,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-20 02:55:38,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 300903cec0fd231ca88313bcf649662a, server=jenkins-hbase4.apache.org,39331,1689821732285 in 219 msec 2023-07-20 02:55:38,884 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-20 02:55:38,886 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=300903cec0fd231ca88313bcf649662a, ASSIGN in 382 msec 2023-07-20 02:55:38,887 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 02:55:38,887 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821738887"}]},"ts":"1689821738887"} 2023-07-20 02:55:38,889 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-20 02:55:38,892 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 02:55:38,894 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 483 msec 2023-07-20 02:55:39,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 02:55:39,049 INFO [Listener at localhost/46633] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-20 02:55:39,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:39,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-20 02:55:39,055 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 02:55:39,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-20 02:55:39,057 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 02:55:39,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 02:55:39,059 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:39,060 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb empty. 2023-07-20 02:55:39,060 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:39,060 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-20 02:55:39,079 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-20 02:55:39,080 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 392fe9dfffd6a3888f97c341c2952aeb, NAME => 't2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.tmp 2023-07-20 02:55:39,099 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:39,099 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 392fe9dfffd6a3888f97c341c2952aeb, disabling compactions & flushes 2023-07-20 02:55:39,099 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:39,099 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:39,099 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. after waiting 0 ms 2023-07-20 02:55:39,099 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:39,099 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:39,099 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 392fe9dfffd6a3888f97c341c2952aeb: 2023-07-20 02:55:39,103 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 02:55:39,106 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821739105"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689821739105"}]},"ts":"1689821739105"} 2023-07-20 02:55:39,108 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 02:55:39,109 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 02:55:39,110 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821739109"}]},"ts":"1689821739109"} 2023-07-20 02:55:39,111 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-20 02:55:39,119 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 02:55:39,121 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 02:55:39,121 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 02:55:39,121 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 02:55:39,121 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=392fe9dfffd6a3888f97c341c2952aeb, ASSIGN}] 2023-07-20 02:55:39,123 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=392fe9dfffd6a3888f97c341c2952aeb, ASSIGN 2023-07-20 02:55:39,124 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=392fe9dfffd6a3888f97c341c2952aeb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46185,1689821732333; forceNewPlan=false, retain=false 2023-07-20 02:55:39,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 02:55:39,275 INFO [jenkins-hbase4:36235] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 02:55:39,276 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=392fe9dfffd6a3888f97c341c2952aeb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:39,277 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821739276"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821739276"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821739276"}]},"ts":"1689821739276"} 2023-07-20 02:55:39,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 392fe9dfffd6a3888f97c341c2952aeb, server=jenkins-hbase4.apache.org,46185,1689821732333}] 2023-07-20 02:55:39,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 02:55:39,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:39,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 392fe9dfffd6a3888f97c341c2952aeb, NAME => 't2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:39,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:39,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:39,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:39,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:39,445 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:39,449 DEBUG [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/cf1 2023-07-20 02:55:39,449 DEBUG [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/cf1 2023-07-20 02:55:39,449 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 392fe9dfffd6a3888f97c341c2952aeb columnFamilyName cf1 2023-07-20 02:55:39,451 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] regionserver.HStore(310): Store=392fe9dfffd6a3888f97c341c2952aeb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:39,451 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:39,453 DEBUG [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/norep 2023-07-20 02:55:39,453 DEBUG [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/norep 2023-07-20 02:55:39,453 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 392fe9dfffd6a3888f97c341c2952aeb columnFamilyName norep 2023-07-20 02:55:39,455 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] regionserver.HStore(310): Store=392fe9dfffd6a3888f97c341c2952aeb/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:39,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:39,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:39,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 02:55:39,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:39,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 02:55:39,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 392fe9dfffd6a3888f97c341c2952aeb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73764524, jitterRate=0.09917706251144409}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 02:55:39,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 392fe9dfffd6a3888f97c341c2952aeb: 2023-07-20 02:55:39,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb., pid=14, masterSystemTime=1689821739434 2023-07-20 02:55:39,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:39,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:39,474 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=392fe9dfffd6a3888f97c341c2952aeb, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:39,474 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821739474"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821739474"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821739474"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821739474"}]},"ts":"1689821739474"} 2023-07-20 02:55:39,480 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-20 02:55:39,480 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 392fe9dfffd6a3888f97c341c2952aeb, server=jenkins-hbase4.apache.org,46185,1689821732333 in 197 msec 2023-07-20 02:55:39,486 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-20 02:55:39,486 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=392fe9dfffd6a3888f97c341c2952aeb, ASSIGN in 359 msec 2023-07-20 02:55:39,487 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 02:55:39,487 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821739487"}]},"ts":"1689821739487"} 2023-07-20 02:55:39,489 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-20 02:55:39,492 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 02:55:39,494 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 441 msec 2023-07-20 02:55:39,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 02:55:39,667 INFO [Listener at localhost/46633] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-20 02:55:39,670 DEBUG [Listener at localhost/46633] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-20 02:55:39,672 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36432, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-20 02:55:39,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:39,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-20 02:55:39,679 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 02:55:39,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-20 02:55:39,681 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 02:55:39,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 02:55:39,682 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 02:55:39,684 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:39,684 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544 empty. 2023-07-20 02:55:39,685 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:39,685 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-20 02:55:39,719 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-20 02:55:39,721 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 80b6625deaec89ca31a4c1b65e4a6544, NAME => 't1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp 2023-07-20 02:55:39,746 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:39,746 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 80b6625deaec89ca31a4c1b65e4a6544, disabling compactions & flushes 2023-07-20 02:55:39,746 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:39,746 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:39,746 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. after waiting 0 ms 2023-07-20 02:55:39,746 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:39,746 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:39,746 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 80b6625deaec89ca31a4c1b65e4a6544: 2023-07-20 02:55:39,750 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 02:55:39,752 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821739752"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689821739752"}]},"ts":"1689821739752"} 2023-07-20 02:55:39,755 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 02:55:39,757 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 02:55:39,758 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821739757"}]},"ts":"1689821739757"} 2023-07-20 02:55:39,760 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-20 02:55:39,769 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 02:55:39,775 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 02:55:39,775 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 02:55:39,775 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-20 02:55:39,775 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-20 02:55:39,775 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 02:55:39,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN}] 2023-07-20 02:55:39,777 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN 2023-07-20 02:55:39,778 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44133,1689821736661; forceNewPlan=false, retain=false 2023-07-20 02:55:39,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 02:55:39,850 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-20 02:55:39,851 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-20 02:55:39,852 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-20 02:55:39,852 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-20 02:55:39,929 INFO [jenkins-hbase4:45829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 02:55:39,930 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=80b6625deaec89ca31a4c1b65e4a6544, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:39,931 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821739930"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821739930"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821739930"}]},"ts":"1689821739930"} 2023-07-20 02:55:39,933 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 80b6625deaec89ca31a4c1b65e4a6544, server=jenkins-hbase4.apache.org,44133,1689821736661}] 2023-07-20 02:55:39,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 02:55:40,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:40,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 80b6625deaec89ca31a4c1b65e4a6544, NAME => 't1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:40,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:40,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:40,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:40,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:40,093 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:40,095 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1 2023-07-20 02:55:40,095 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1 2023-07-20 02:55:40,096 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80b6625deaec89ca31a4c1b65e4a6544 columnFamilyName cf1 2023-07-20 02:55:40,096 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(310): Store=80b6625deaec89ca31a4c1b65e4a6544/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:40,096 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:40,098 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/norep 2023-07-20 02:55:40,098 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/norep 2023-07-20 02:55:40,099 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80b6625deaec89ca31a4c1b65e4a6544 columnFamilyName norep 2023-07-20 02:55:40,099 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(310): Store=80b6625deaec89ca31a4c1b65e4a6544/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:40,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:40,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:40,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 02:55:40,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:40,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 02:55:40,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 80b6625deaec89ca31a4c1b65e4a6544; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64030329, jitterRate=-0.045873746275901794}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 02:55:40,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 80b6625deaec89ca31a4c1b65e4a6544: 2023-07-20 02:55:40,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544., pid=11, masterSystemTime=1689821740086 2023-07-20 02:55:40,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:40,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:40,112 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=80b6625deaec89ca31a4c1b65e4a6544, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:40,113 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821740112"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821740112"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821740112"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821740112"}]},"ts":"1689821740112"} 2023-07-20 02:55:40,117 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-20 02:55:40,117 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 80b6625deaec89ca31a4c1b65e4a6544, server=jenkins-hbase4.apache.org,44133,1689821736661 in 181 msec 2023-07-20 02:55:40,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-20 02:55:40,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN in 342 msec 2023-07-20 02:55:40,121 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 02:55:40,121 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821740121"}]},"ts":"1689821740121"} 2023-07-20 02:55:40,123 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-20 02:55:40,131 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 02:55:40,133 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 456 msec 2023-07-20 02:55:40,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 02:55:40,286 INFO [Listener at localhost/46633] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-20 02:55:40,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:40,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-20 02:55:40,292 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 02:55:40,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-20 02:55:40,294 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 02:55:40,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 02:55:40,296 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/default/t2_syncup/6908f43230333d97246933b2af61e211 2023-07-20 02:55:40,297 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/default/t2_syncup/6908f43230333d97246933b2af61e211 empty. 2023-07-20 02:55:40,297 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/default/t2_syncup/6908f43230333d97246933b2af61e211 2023-07-20 02:55:40,297 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-20 02:55:40,319 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-20 02:55:40,321 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6908f43230333d97246933b2af61e211, NAME => 't2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.tmp 2023-07-20 02:55:40,338 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:40,338 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 6908f43230333d97246933b2af61e211, disabling compactions & flushes 2023-07-20 02:55:40,338 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:40,338 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:40,338 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. after waiting 0 ms 2023-07-20 02:55:40,338 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:40,338 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:40,338 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 6908f43230333d97246933b2af61e211: 2023-07-20 02:55:40,341 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 02:55:40,344 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821740343"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689821740343"}]},"ts":"1689821740343"} 2023-07-20 02:55:40,346 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 02:55:40,348 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 02:55:40,348 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821740348"}]},"ts":"1689821740348"} 2023-07-20 02:55:40,350 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-20 02:55:40,355 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 02:55:40,357 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 02:55:40,357 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 02:55:40,357 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-20 02:55:40,357 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-20 02:55:40,357 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 02:55:40,357 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN}] 2023-07-20 02:55:40,360 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN 2023-07-20 02:55:40,362 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40959,1689821736691; forceNewPlan=false, retain=false 2023-07-20 02:55:40,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 02:55:40,512 INFO [jenkins-hbase4:45829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 02:55:40,513 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=6908f43230333d97246933b2af61e211, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:40,514 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821740513"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821740513"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821740513"}]},"ts":"1689821740513"} 2023-07-20 02:55:40,516 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 6908f43230333d97246933b2af61e211, server=jenkins-hbase4.apache.org,40959,1689821736691}] 2023-07-20 02:55:40,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 02:55:40,670 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:40,670 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:40,672 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49552, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:55:40,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:40,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6908f43230333d97246933b2af61e211, NAME => 't2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:40,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 6908f43230333d97246933b2af61e211 2023-07-20 02:55:40,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:40,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6908f43230333d97246933b2af61e211 2023-07-20 02:55:40,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6908f43230333d97246933b2af61e211 2023-07-20 02:55:40,679 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6908f43230333d97246933b2af61e211 2023-07-20 02:55:40,681 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1 2023-07-20 02:55:40,681 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1 2023-07-20 02:55:40,681 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6908f43230333d97246933b2af61e211 columnFamilyName cf1 2023-07-20 02:55:40,682 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(310): Store=6908f43230333d97246933b2af61e211/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:40,682 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 6908f43230333d97246933b2af61e211 2023-07-20 02:55:40,683 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/norep 2023-07-20 02:55:40,683 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/norep 2023-07-20 02:55:40,684 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6908f43230333d97246933b2af61e211 columnFamilyName norep 2023-07-20 02:55:40,685 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(310): Store=6908f43230333d97246933b2af61e211/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:40,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211 2023-07-20 02:55:40,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211 2023-07-20 02:55:40,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 02:55:40,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6908f43230333d97246933b2af61e211 2023-07-20 02:55:40,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 02:55:40,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6908f43230333d97246933b2af61e211; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67695759, jitterRate=0.00874541699886322}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 02:55:40,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6908f43230333d97246933b2af61e211: 2023-07-20 02:55:40,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211., pid=14, masterSystemTime=1689821740670 2023-07-20 02:55:40,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:40,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:40,704 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=6908f43230333d97246933b2af61e211, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:40,704 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821740704"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821740704"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821740704"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821740704"}]},"ts":"1689821740704"} 2023-07-20 02:55:40,709 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-20 02:55:40,709 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 6908f43230333d97246933b2af61e211, server=jenkins-hbase4.apache.org,40959,1689821736691 in 190 msec 2023-07-20 02:55:40,711 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-20 02:55:40,711 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN in 352 msec 2023-07-20 02:55:40,712 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 02:55:40,712 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689821740712"}]},"ts":"1689821740712"} 2023-07-20 02:55:40,713 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-20 02:55:40,716 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 02:55:40,718 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 428 msec 2023-07-20 02:55:40,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45829] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 02:55:40,899 INFO [Listener at localhost/46633] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-20 02:55:40,900 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(139): Connect 0x6b43e924 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:40,909 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7202ca53, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:40,910 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(139): Connect 0x6b831c8f to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:40,918 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1ffdd212, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:40,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.HMaster(3464): Client=jenkins//172.31.14.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:55129:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-07-20 02:55:40,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-20 02:55:40,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-20 02:55:40,958 INFO [PEWorker-1] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 02:55:40,958 INFO [PEWorker-1] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@47c0bb8echeck-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:40,962 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@47c0bb8echeck-peer-cluster-id0x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:40,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@47c0bb8echeck-peer-cluster-id-0x10180a7ca410014 connected 2023-07-20 02:55:41,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-20 02:55:41,065 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@47c0bb8echeck-peer-cluster-id-0x10180a7ca410014, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:41,079 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:41,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-07-20 02:55:41,200 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 02:55:41,200 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-20 02:55:41,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-20 02:55:41,257 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39331] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-07-20 02:55:41,257 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46185] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-07-20 02:55:41,259 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-20 02:55:41,260 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-20 02:55:41,272 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 02:55:41,272 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 02:55:41,292 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-20 02:55:41,292 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-20 02:55:41,320 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-07-20 02:55:41,320 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-07-20 02:55:41,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36235] master.HMaster(3641): Remote procedure done, pid=16 2023-07-20 02:55:41,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] master.HMaster(3641): Remote procedure done, pid=17 2023-07-20 02:55:41,323 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36235] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,39331,1689821732285 suceeded 2023-07-20 02:55:41,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36235] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,46185,1689821732333 suceeded 2023-07-20 02:55:41,329 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 236 msec 2023-07-20 02:55:41,330 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-07-20 02:55:41,330 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 236 msec 2023-07-20 02:55:41,330 INFO [PEWorker-3] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:55129:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-07-20 02:55:41,332 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 396 msec 2023-07-20 02:55:41,348 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x1cf903ed to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:41,348 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x5c078677 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:41,355 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1521ae05, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:41,355 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@7c76bd58, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:41,360 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:41,360 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:41,364 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:41,365 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10180a7ca410017 connected 2023-07-20 02:55:41,366 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:41,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10180a7ca410018 connected 2023-07-20 02:55:41,367 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=1) is replicating from cluster=bbd9fa59-2213-4359-9606-c616f1cd40eb to cluster=69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:55:41,368 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=1) is replicating from cluster=bbd9fa59-2213-4359-9606-c616f1cd40eb to cluster=69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:55:41,370 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C39331%2C1689821732285 2023-07-20 02:55:41,370 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C46185%2C1689821732333 2023-07-20 02:55:41,375 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 02:55:41,375 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 02:55:41,379 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C39331%2C1689821732285 2023-07-20 02:55:41,379 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C46185%2C1689821732333 2023-07-20 02:55:41,459 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-20 02:55:41,459 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 02:55:41,459 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-20 02:55:41,459 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-20 02:55:41,459 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 02:55:41,459 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 02:55:41,459 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822, lastWalPosition=901, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 02:55:41,459 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-20 02:55:41,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36235] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-20 02:55:41,553 DEBUG [Listener at localhost/46633] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-07-20 02:55:41,555 DEBUG [hconnection-0x6e2d32d5-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:41,557 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53226, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:41,562 DEBUG [Listener at localhost/46633] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:41,564 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60894, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:41,574 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-20 02:55:41,576 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 487 2023-07-20 02:55:41,576 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-20 02:55:41,576 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 487 2023-07-20 02:55:41,576 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/300903cec0fd231ca88313bcf649662a/4=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822, lastWalPosition=487, nbRowKeys=1, nbHFiles=0, heapSize=208, lastSeqIds={}, endOfFile=false,usedBufferSize=208] 2023-07-20 02:55:41,593 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:41,596 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47844, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-20 02:55:41,602 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44133] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 02:55:41,603 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44133] zookeeper.ReadOnlyZKClient(139): Connect 0x5bd5cd7b to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:41,609 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44133] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4945927a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:41,619 DEBUG [hconnection-0xa5ada68-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:41,620 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50186, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-20 02:55:41,646 DEBUG [hconnection-0xa5ada68-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:41,647 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47846, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-20 02:55:41,671 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44133] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 02:55:41,682 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-20 02:55:41,696 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 7399 2023-07-20 02:55:41,696 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceWALReader(162): Read 66 WAL entries eligible for replication 2023-07-20 02:55:41,696 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 7399 2023-07-20 02:55:41,697 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/300903cec0fd231ca88313bcf649662a/5=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/6=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/7=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/8=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/9=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/10=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/11=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/12=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/13=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/14=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/15=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/16=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/17=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/18=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/19=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/20=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/21=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/22=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/23=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/24=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/25=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/26=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/27=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/28=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/29=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/30=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/31=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/32=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/33=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/34=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/35=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/36=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/37=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/38=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/39=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/40=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/41=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/42=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/43=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/44=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/45=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/46=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/47=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/48=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/49=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/50=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/51=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/52=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/53=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/54=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/55=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/56=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/57=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/58=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/59=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/60=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/61=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/62=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/63=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/64=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/65=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/66=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/67=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/68=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/69=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/70=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822, lastWalPosition=7399, nbRowKeys=66, nbHFiles=0, heapSize=13728, lastSeqIds={}, endOfFile=false,usedBufferSize=13728] 2023-07-20 02:55:41,705 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44133] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 02:55:41,714 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44133] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 02:55:41,802 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 5606 2023-07-20 02:55:41,802 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceWALReader(162): Read 45 WAL entries eligible for replication 2023-07-20 02:55:41,804 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 5606 2023-07-20 02:55:41,805 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/4=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/5=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/6=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/7=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/8=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/9=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/10=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/11=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/12=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/13=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/14=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/15=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/16=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/17=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/18=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/19=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/20=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/21=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/22=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/23=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/24=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/25=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/26=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/27=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/28=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/29=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/30=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/31=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/32=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/33=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/34=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/35=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/36=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/37=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/38=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/39=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/40=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/41=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/42=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/43=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/44=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/45=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/46=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/47=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/48=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822, lastWalPosition=5606, nbRowKeys=45, nbHFiles=0, heapSize=9360, lastSeqIds={}, endOfFile=false,usedBufferSize=9360] 2023-07-20 02:55:41,810 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:41,811 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50198, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-20 02:55:41,813 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46489] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 02:55:41,814 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:41,814 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceWALReader(162): Read 33 WAL entries eligible for replication 2023-07-20 02:55:41,814 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:41,814 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/300903cec0fd231ca88313bcf649662a/71=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/72=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/73=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/74=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/75=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/76=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/77=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/78=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/79=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/80=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/81=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/82=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/83=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/84=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/85=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/86=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/87=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/88=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/89=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/90=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/91=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/92=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/93=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/94=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/95=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/96=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/97=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/98=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/99=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/100=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/101=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/102=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822, lastWalPosition=10975, nbRowKeys=33, nbHFiles=0, heapSize=6864, lastSeqIds={}, endOfFile=false,usedBufferSize=6864] 2023-07-20 02:55:41,815 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46489] zookeeper.ReadOnlyZKClient(139): Connect 0x5d3988e3 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:41,820 DEBUG [SinkThread-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:41,822 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50200, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-20 02:55:41,838 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=46489] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 02:55:41,840 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5404d95a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:41,844 DEBUG [hconnection-0x6c2e9340-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:41,845 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50204, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-20 02:55:41,851 DEBUG [hconnection-0x6c2e9340-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:41,854 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49560, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-20 02:55:41,855 DEBUG [hconnection-0x6c2e9340-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:41,857 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47862, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-20 02:55:41,860 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46489] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 02:55:41,861 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=46489] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 02:55:41,922 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:41,928 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 13414 2023-07-20 02:55:41,929 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceWALReader(162): Read 74 WAL entries eligible for replication 2023-07-20 02:55:41,929 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/49=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/50=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/51=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/52=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/53=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/54=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/55=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/56=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/57=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/58=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/59=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/60=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/61=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/62=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/63=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/64=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/65=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/66=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/67=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/68=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/69=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/70=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/71=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/72=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/73=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/74=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/75=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/76=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/77=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/78=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/79=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/80=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/81=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/82=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/83=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/84=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/85=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/86=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/87=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/88=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/89=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/90=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/91=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/92=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/93=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/94=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/95=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/96=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/97=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/98=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/99=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/100=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/101=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/102=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/103=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/104=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/105=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/106=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/107=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/108=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/109=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/110=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/111=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/112=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/113=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/114=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/115=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/116=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/117=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/118=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/119=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/120=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/121=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/122=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822, lastWalPosition=13414, nbRowKeys=74, nbHFiles=0, heapSize=15392, lastSeqIds={}, endOfFile=false,usedBufferSize=15392] 2023-07-20 02:55:41,930 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 13414 2023-07-20 02:55:41,933 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:41,936 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39288, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-20 02:55:41,938 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38819] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 02:55:41,938 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38819] zookeeper.ReadOnlyZKClient(139): Connect 0x157ea2a5 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:41,945 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38819] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1bb94e7a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:41,948 DEBUG [hconnection-0x1f86c9d5-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:41,949 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50214, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-20 02:55:41,958 DEBUG [hconnection-0x1f86c9d5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:41,960 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49568, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-20 02:55:41,966 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38819] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 02:55:42,034 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:42,051 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 21833 2023-07-20 02:55:42,051 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceWALReader(162): Read 78 WAL entries eligible for replication 2023-07-20 02:55:42,051 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 21833 2023-07-20 02:55:42,052 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/123=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/124=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/125=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/126=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/127=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/128=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/129=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/130=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/131=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/132=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/133=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/134=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/135=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/136=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/137=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/138=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/139=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/140=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/141=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/142=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/143=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/144=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/145=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/146=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/147=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/148=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/149=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/150=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/151=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/152=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/153=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/154=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/155=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/156=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/157=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/158=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/159=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/160=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/161=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/162=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/163=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/164=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/165=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/166=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/167=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/168=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/169=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/170=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/171=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/172=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/173=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/174=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/175=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/176=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/177=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/178=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/179=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/180=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/181=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/182=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/183=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/184=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/185=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/186=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/187=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/188=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/189=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/190=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/191=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/192=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/193=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/194=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/195=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/196=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/197=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/198=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/199=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/200=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822, lastWalPosition=21833, nbRowKeys=78, nbHFiles=0, heapSize=16224, lastSeqIds={}, endOfFile=false,usedBufferSize=16224] 2023-07-20 02:55:42,056 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=46489] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 02:55:42,063 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=46489] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 02:55:42,142 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:42,160 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:42,160 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceWALReader(162): Read 3 WAL entries eligible for replication 2023-07-20 02:55:42,160 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:42,160 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/201=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/202=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822, lastWalPosition=22269, nbRowKeys=3, nbHFiles=0, heapSize=624, lastSeqIds={}, endOfFile=false,usedBufferSize=624] 2023-07-20 02:55:42,160 DEBUG [SinkThread-3] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:42,162 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49578, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-20 02:55:42,163 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40959] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 02:55:42,163 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40959] zookeeper.ReadOnlyZKClient(139): Connect 0x44aa0b76 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:42,169 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40959] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4a6ded93, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:42,171 DEBUG [hconnection-0x4e526006-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:42,172 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50218, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-20 02:55:42,176 DEBUG [hconnection-0x4e526006-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:42,177 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49584, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-20 02:55:42,179 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40959] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 02:55:42,213 INFO [jenkins-hbase4:40959Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:55:42,213 INFO [jenkins-hbase4:40959Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 3 2023-07-20 02:55:42,215 INFO [jenkins-hbase4:46489Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:55:42,216 INFO [jenkins-hbase4:46489Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 156 2023-07-20 02:55:42,219 INFO [jenkins-hbase4:38819Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:55:42,219 INFO [jenkins-hbase4:38819Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 74 2023-07-20 02:55:42,222 INFO [jenkins-hbase4:44133Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:55:42,222 INFO [jenkins-hbase4:44133Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 67 2023-07-20 02:55:42,250 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:42,267 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:42,358 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:42,375 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:42,466 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:42,488 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:42,557 DEBUG [hconnection-0x7b49d3ec-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:42,559 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50234, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:42,564 DEBUG [hconnection-0x7b49d3ec-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:42,565 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47870, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:42,573 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:42,583 DEBUG [hconnection-0x7b49d3ec-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:42,585 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49592, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:42,592 DEBUG [Listener at localhost/46633] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-07-20 02:55:42,592 INFO [Listener at localhost/46633] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-20 02:55:42,592 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x607cb094 to 127.0.0.1:55129 2023-07-20 02:55:42,593 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:42,598 DEBUG [Listener at localhost/46633] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-20 02:55:42,598 DEBUG [Listener at localhost/46633] util.JVMClusterUtil(257): Found active master hash=1372053114, stopped=false 2023-07-20 02:55:42,598 INFO [Listener at localhost/46633] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,45829,1689821736619 2023-07-20 02:55:42,600 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:42,600 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 02:55:42,600 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 02:55:42,600 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 02:55:42,600 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 02:55:42,600 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 02:55:42,600 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:42,601 INFO [Listener at localhost/46633] procedure2.ProcedureExecutor(629): Stopping 2023-07-20 02:55:42,601 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:42,601 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:42,601 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:42,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:42,601 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:42,601 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x21722d70 to 127.0.0.1:55129 2023-07-20 02:55:42,603 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:42,603 INFO [Listener at localhost/46633] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44133,1689821736661' ***** 2023-07-20 02:55:42,603 INFO [Listener at localhost/46633] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 02:55:42,603 INFO [Listener at localhost/46633] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40959,1689821736691' ***** 2023-07-20 02:55:42,603 INFO [Listener at localhost/46633] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 02:55:42,604 INFO [RS:0;jenkins-hbase4:44133] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 02:55:42,604 INFO [RS:1;jenkins-hbase4:40959] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 02:55:42,604 INFO [Listener at localhost/46633] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38819,1689821736718' ***** 2023-07-20 02:55:42,604 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 02:55:42,604 INFO [RS:1;jenkins-hbase4:40959] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 02:55:42,604 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 02:55:42,605 INFO [RS:1;jenkins-hbase4:40959] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 02:55:42,604 INFO [RS:0;jenkins-hbase4:44133] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 02:55:42,607 INFO [RS:0;jenkins-hbase4:44133] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 02:55:42,604 INFO [Listener at localhost/46633] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 02:55:42,607 INFO [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(3305): Received CLOSE for 6908f43230333d97246933b2af61e211 2023-07-20 02:55:42,607 INFO [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(3305): Received CLOSE for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:42,607 INFO [RS:2;jenkins-hbase4:38819] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 02:55:42,607 INFO [Listener at localhost/46633] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46489,1689821736748' ***** 2023-07-20 02:55:42,607 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 02:55:42,607 INFO [Listener at localhost/46633] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 02:55:42,607 INFO [RS:2;jenkins-hbase4:38819] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 02:55:42,607 INFO [RS:2;jenkins-hbase4:38819] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 02:55:42,608 INFO [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:42,608 INFO [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:42,608 DEBUG [RS:1;jenkins-hbase4:40959] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3e3197a8 to 127.0.0.1:55129 2023-07-20 02:55:42,608 DEBUG [RS:2;jenkins-hbase4:38819] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c28d054 to 127.0.0.1:55129 2023-07-20 02:55:42,608 DEBUG [RS:1;jenkins-hbase4:40959] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:42,608 DEBUG [RS:2;jenkins-hbase4:38819] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:42,608 INFO [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-20 02:55:42,608 INFO [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(3305): Received CLOSE for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:42,608 INFO [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38819,1689821736718; all regions closed. 2023-07-20 02:55:42,609 INFO [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:42,608 DEBUG [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1478): Online Regions={6908f43230333d97246933b2af61e211=t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.} 2023-07-20 02:55:42,609 DEBUG [RS:0;jenkins-hbase4:44133] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2032c975 to 127.0.0.1:55129 2023-07-20 02:55:42,609 DEBUG [RS:0;jenkins-hbase4:44133] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:42,609 INFO [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-20 02:55:42,609 DEBUG [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1478): Online Regions={956fe6fd22cc055b7d5c09ea7e2fd4a7=hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7., 80b6625deaec89ca31a4c1b65e4a6544=t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.} 2023-07-20 02:55:42,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6908f43230333d97246933b2af61e211, disabling compactions & flushes 2023-07-20 02:55:42,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:42,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:42,612 INFO [RS:3;jenkins-hbase4:46489] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 02:55:42,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. after waiting 0 ms 2023-07-20 02:55:42,612 INFO [RS:3;jenkins-hbase4:46489] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 02:55:42,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:42,612 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 02:55:42,612 INFO [RS:3;jenkins-hbase4:46489] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 02:55:42,612 DEBUG [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1504): Waiting on 80b6625deaec89ca31a4c1b65e4a6544, 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:42,612 DEBUG [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1504): Waiting on 6908f43230333d97246933b2af61e211 2023-07-20 02:55:42,613 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 956fe6fd22cc055b7d5c09ea7e2fd4a7, disabling compactions & flushes 2023-07-20 02:55:42,612 INFO [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:42,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:42,614 DEBUG [RS:3;jenkins-hbase4:46489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x54ccb3c5 to 127.0.0.1:55129 2023-07-20 02:55:42,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:42,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. after waiting 0 ms 2023-07-20 02:55:42,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:42,614 DEBUG [RS:3;jenkins-hbase4:46489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:42,614 INFO [RS:3;jenkins-hbase4:46489] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 02:55:42,614 INFO [RS:3;jenkins-hbase4:46489] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 02:55:42,614 INFO [RS:3;jenkins-hbase4:46489] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 02:55:42,614 INFO [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-20 02:55:42,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 956fe6fd22cc055b7d5c09ea7e2fd4a7 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-20 02:55:42,614 INFO [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-20 02:55:42,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6908f43230333d97246933b2af61e211 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-20 02:55:42,615 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-07-20 02:55:42,615 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-20 02:55:42,619 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 02:55:42,619 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 02:55:42,619 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 02:55:42,619 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 02:55:42,619 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 02:55:42,620 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-07-20 02:55:42,631 DEBUG [RS:2;jenkins-hbase4:38819] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs 2023-07-20 02:55:42,631 INFO [RS:2;jenkins-hbase4:38819] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38819%2C1689821736718:(num 1689821737328) 2023-07-20 02:55:42,631 DEBUG [RS:2;jenkins-hbase4:38819] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:42,631 INFO [RS:2;jenkins-hbase4:38819] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:42,631 INFO [RS:2;jenkins-hbase4:38819] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-20 02:55:42,631 INFO [RS:2;jenkins-hbase4:38819] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 02:55:42,632 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:55:42,632 INFO [RS:2;jenkins-hbase4:38819] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 02:55:42,632 INFO [RS:2;jenkins-hbase4:38819] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 02:55:42,632 DEBUG [RS:2;jenkins-hbase4:38819] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x157ea2a5 to 127.0.0.1:55129 2023-07-20 02:55:42,632 DEBUG [RS:2;jenkins-hbase4:38819] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:42,634 INFO [RS:2;jenkins-hbase4:38819] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38819 2023-07-20 02:55:42,646 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:42,646 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:42,646 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:42,646 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:42,646 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410017, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:42,646 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:42,646 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:55:42,646 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410018, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:42,646 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:42,646 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:55:42,646 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:42,646 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,38819,1689821736718 2023-07-20 02:55:42,646 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:42,648 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38819,1689821736718] 2023-07-20 02:55:42,648 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38819,1689821736718; numProcessing=1 2023-07-20 02:55:42,650 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,38819,1689821736718 already deleted, retry=false 2023-07-20 02:55:42,650 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38819,1689821736718 expired; onlineServers=3 2023-07-20 02:55:42,683 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:42,700 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:42,700 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:42,700 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:42,701 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:42,724 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:42,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=8 (bloomFilter=true), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/.tmp/cf1/a2587aab5db342319f7ea6dadbb7933e 2023-07-20 02:55:42,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/.tmp/info/ba087245ad7f4c18b071b799180f415c 2023-07-20 02:55:42,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/.tmp/info/ba087245ad7f4c18b071b799180f415c as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/info/ba087245ad7f4c18b071b799180f415c 2023-07-20 02:55:42,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/.tmp/cf1/a2587aab5db342319f7ea6dadbb7933e as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1/a2587aab5db342319f7ea6dadbb7933e 2023-07-20 02:55:42,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1/a2587aab5db342319f7ea6dadbb7933e, entries=200, sequenceid=8, filesize=12.2 K 2023-07-20 02:55:42,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/info/ba087245ad7f4c18b071b799180f415c, entries=2, sequenceid=6, filesize=4.8 K 2023-07-20 02:55:42,791 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:42,794 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 956fe6fd22cc055b7d5c09ea7e2fd4a7 in 180ms, sequenceid=6, compaction requested=false 2023-07-20 02:55:42,794 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 6908f43230333d97246933b2af61e211 in 180ms, sequenceid=8, compaction requested=false 2023-07-20 02:55:42,798 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:42,798 INFO [RS:2;jenkins-hbase4:38819] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38819,1689821736718; zookeeper connection closed. 2023-07-20 02:55:42,798 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38819-0x10180a7ca41000a, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:42,799 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1787ddd0] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1787ddd0 2023-07-20 02:55:42,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/recovered.edits/11.seqid, newMaxSeqId=11, maxSeqId=1 2023-07-20 02:55:42,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-20 02:55:42,811 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:42,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6908f43230333d97246933b2af61e211: 2023-07-20 02:55:42,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:42,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:42,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 956fe6fd22cc055b7d5c09ea7e2fd4a7: 2023-07-20 02:55:42,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:42,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 80b6625deaec89ca31a4c1b65e4a6544, disabling compactions & flushes 2023-07-20 02:55:42,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:42,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:42,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. after waiting 0 ms 2023-07-20 02:55:42,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:42,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 80b6625deaec89ca31a4c1b65e4a6544 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-20 02:55:42,814 INFO [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40959,1689821736691; all regions closed. 2023-07-20 02:55:42,814 DEBUG [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1504): Waiting on 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:42,820 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-20 02:55:42,822 DEBUG [RS:1;jenkins-hbase4:40959] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs 2023-07-20 02:55:42,823 INFO [RS:1;jenkins-hbase4:40959] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40959%2C1689821736691:(num 1689821737321) 2023-07-20 02:55:42,823 DEBUG [RS:1;jenkins-hbase4:40959] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:42,823 INFO [RS:1;jenkins-hbase4:40959] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:42,824 INFO [RS:1;jenkins-hbase4:40959] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 02:55:42,825 INFO [RS:1;jenkins-hbase4:40959] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 02:55:42,825 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:55:42,825 INFO [RS:1;jenkins-hbase4:40959] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 02:55:42,825 INFO [RS:1;jenkins-hbase4:40959] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 02:55:42,826 DEBUG [RS:1;jenkins-hbase4:40959] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x44aa0b76 to 127.0.0.1:55129 2023-07-20 02:55:42,826 DEBUG [RS:1;jenkins-hbase4:40959] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:42,827 INFO [RS:1;jenkins-hbase4:40959] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40959 2023-07-20 02:55:42,834 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:42,834 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:42,834 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410018, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:42,835 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:55:42,835 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,40959,1689821736691 2023-07-20 02:55:42,835 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410017, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:42,835 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:42,836 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:55:42,837 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40959,1689821736691] 2023-07-20 02:55:42,837 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40959,1689821736691; numProcessing=2 2023-07-20 02:55:42,837 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:42,838 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,40959,1689821736691 already deleted, retry=false 2023-07-20 02:55:42,838 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40959,1689821736691 expired; onlineServers=2 2023-07-20 02:55:42,843 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/.tmp/cf1/db3cbecb4ef74734b4315fa2f2bf0702 2023-07-20 02:55:42,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/.tmp/cf1/db3cbecb4ef74734b4315fa2f2bf0702 as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1/db3cbecb4ef74734b4315fa2f2bf0702 2023-07-20 02:55:42,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1/db3cbecb4ef74734b4315fa2f2bf0702, entries=100, sequenceid=7, filesize=8.3 K 2023-07-20 02:55:42,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 80b6625deaec89ca31a4c1b65e4a6544 in 47ms, sequenceid=7, compaction requested=false 2023-07-20 02:55:42,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-20 02:55:42,868 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:42,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 80b6625deaec89ca31a4c1b65e4a6544: 2023-07-20 02:55:42,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:42,899 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:42,944 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:43,014 INFO [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44133,1689821736661; all regions closed. 2023-07-20 02:55:43,015 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:43,020 DEBUG [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-20 02:55:43,025 DEBUG [RS:0;jenkins-hbase4:44133] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs 2023-07-20 02:55:43,025 INFO [RS:0;jenkins-hbase4:44133] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44133%2C1689821736661:(num 1689821737342) 2023-07-20 02:55:43,025 DEBUG [RS:0;jenkins-hbase4:44133] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:43,025 INFO [RS:0;jenkins-hbase4:44133] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:43,025 INFO [RS:0;jenkins-hbase4:44133] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 02:55:43,026 INFO [RS:0;jenkins-hbase4:44133] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 02:55:43,026 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:55:43,026 INFO [RS:0;jenkins-hbase4:44133] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 02:55:43,026 INFO [RS:0;jenkins-hbase4:44133] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 02:55:43,026 DEBUG [RS:0;jenkins-hbase4:44133] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5bd5cd7b to 127.0.0.1:55129 2023-07-20 02:55:43,027 DEBUG [RS:0;jenkins-hbase4:44133] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:43,028 INFO [RS:0;jenkins-hbase4:44133] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44133 2023-07-20 02:55:43,031 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:43,031 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410017, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:43,031 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,44133,1689821736661 2023-07-20 02:55:43,031 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:55:43,031 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:43,031 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410018, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:43,031 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:55:43,032 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,44133,1689821736661] 2023-07-20 02:55:43,032 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,44133,1689821736661; numProcessing=3 2023-07-20 02:55:43,033 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,44133,1689821736661 already deleted, retry=false 2023-07-20 02:55:43,033 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,44133,1689821736661 expired; onlineServers=1 2023-07-20 02:55:43,051 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:43,124 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:43,126 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/.tmp/info/441251dfb1734be1aa02887ae7be7817 2023-07-20 02:55:43,147 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/.tmp/table/838ab0139f504a0b99265e300d89842c 2023-07-20 02:55:43,155 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/.tmp/info/441251dfb1734be1aa02887ae7be7817 as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info/441251dfb1734be1aa02887ae7be7817 2023-07-20 02:55:43,158 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:43,163 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info/441251dfb1734be1aa02887ae7be7817, entries=30, sequenceid=19, filesize=8.1 K 2023-07-20 02:55:43,165 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/.tmp/table/838ab0139f504a0b99265e300d89842c as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table/838ab0139f504a0b99265e300d89842c 2023-07-20 02:55:43,172 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table/838ab0139f504a0b99265e300d89842c, entries=6, sequenceid=19, filesize=4.8 K 2023-07-20 02:55:43,173 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 553ms, sequenceid=19, compaction requested=false 2023-07-20 02:55:43,182 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-20 02:55:43,183 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 02:55:43,184 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 02:55:43,184 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 02:55:43,184 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-20 02:55:43,220 INFO [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46489,1689821736748; all regions closed. 2023-07-20 02:55:43,227 DEBUG [RS:3;jenkins-hbase4:46489] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs 2023-07-20 02:55:43,227 INFO [RS:3;jenkins-hbase4:46489] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46489%2C1689821736748.meta:.meta(num 1689821737522) 2023-07-20 02:55:43,234 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:43,235 DEBUG [RS:3;jenkins-hbase4:46489] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs 2023-07-20 02:55:43,235 INFO [RS:3;jenkins-hbase4:46489] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46489%2C1689821736748:(num 1689821737322) 2023-07-20 02:55:43,235 DEBUG [RS:3;jenkins-hbase4:46489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:43,235 INFO [RS:3;jenkins-hbase4:46489] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:43,235 INFO [RS:3;jenkins-hbase4:46489] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 02:55:43,235 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:55:43,236 DEBUG [RS:3;jenkins-hbase4:46489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5d3988e3 to 127.0.0.1:55129 2023-07-20 02:55:43,236 DEBUG [RS:3;jenkins-hbase4:46489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:43,237 INFO [RS:3;jenkins-hbase4:46489] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46489 2023-07-20 02:55:43,240 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46489,1689821736748 2023-07-20 02:55:43,240 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410017, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:43,240 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410018, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:43,240 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:55:43,240 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:55:43,242 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:43,243 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46489,1689821736748] 2023-07-20 02:55:43,243 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46489,1689821736748; numProcessing=4 2023-07-20 02:55:43,244 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46489,1689821736748 already deleted, retry=false 2023-07-20 02:55:43,244 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46489,1689821736748 expired; onlineServers=0 2023-07-20 02:55:43,244 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45829,1689821736619' ***** 2023-07-20 02:55:43,244 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-20 02:55:43,246 DEBUG [M:0;jenkins-hbase4:45829] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@55f8409a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:55:43,246 INFO [M:0;jenkins-hbase4:45829] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45829,1689821736619 2023-07-20 02:55:43,246 INFO [M:0;jenkins-hbase4:45829] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45829,1689821736619; all regions closed. 2023-07-20 02:55:43,246 DEBUG [M:0;jenkins-hbase4:45829] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:43,246 DEBUG [M:0;jenkins-hbase4:45829] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-20 02:55:43,246 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-20 02:55:43,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821736996] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821736996,5,FailOnTimeoutGroup] 2023-07-20 02:55:43,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821736995] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821736995,5,FailOnTimeoutGroup] 2023-07-20 02:55:43,246 DEBUG [M:0;jenkins-hbase4:45829] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-20 02:55:43,249 INFO [M:0;jenkins-hbase4:45829] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-20 02:55:43,249 INFO [M:0;jenkins-hbase4:45829] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-20 02:55:43,249 INFO [M:0;jenkins-hbase4:45829] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-20 02:55:43,249 DEBUG [M:0;jenkins-hbase4:45829] master.HMaster(1512): Stopping service threads 2023-07-20 02:55:43,249 INFO [M:0;jenkins-hbase4:45829] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-20 02:55:43,250 ERROR [M:0;jenkins-hbase4:45829] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-20 02:55:43,250 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-20 02:55:43,250 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:43,251 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:55:43,251 INFO [M:0;jenkins-hbase4:45829] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-20 02:55:43,251 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-20 02:55:43,251 DEBUG [M:0;jenkins-hbase4:45829] zookeeper.ZKUtil(398): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-20 02:55:43,251 WARN [M:0;jenkins-hbase4:45829] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-20 02:55:43,252 INFO [M:0;jenkins-hbase4:45829] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-20 02:55:43,252 INFO [M:0;jenkins-hbase4:45829] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-20 02:55:43,253 DEBUG [M:0;jenkins-hbase4:45829] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 02:55:43,253 INFO [M:0;jenkins-hbase4:45829] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:43,253 DEBUG [M:0;jenkins-hbase4:45829] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:43,253 DEBUG [M:0;jenkins-hbase4:45829] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 02:55:43,253 DEBUG [M:0;jenkins-hbase4:45829] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:43,253 INFO [M:0;jenkins-hbase4:45829] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.40 KB heapSize=65.18 KB 2023-07-20 02:55:43,265 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:43,271 INFO [M:0;jenkins-hbase4:45829] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.40 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7538bca05e2a4ba7834e6fa33e0cf942 2023-07-20 02:55:43,278 DEBUG [M:0;jenkins-hbase4:45829] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7538bca05e2a4ba7834e6fa33e0cf942 as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7538bca05e2a4ba7834e6fa33e0cf942 2023-07-20 02:55:43,284 INFO [M:0;jenkins-hbase4:45829] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7538bca05e2a4ba7834e6fa33e0cf942, entries=14, sequenceid=116, filesize=8.1 K 2023-07-20 02:55:43,285 INFO [M:0;jenkins-hbase4:45829] regionserver.HRegion(2948): Finished flush of dataSize ~55.40 KB/56732, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 32ms, sequenceid=116, compaction requested=false 2023-07-20 02:55:43,287 INFO [M:0;jenkins-hbase4:45829] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:43,287 DEBUG [M:0;jenkins-hbase4:45829] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:55:43,293 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:55:43,293 INFO [M:0;jenkins-hbase4:45829] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-20 02:55:43,294 INFO [M:0;jenkins-hbase4:45829] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45829 2023-07-20 02:55:43,296 DEBUG [M:0;jenkins-hbase4:45829] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,45829,1689821736619 already deleted, retry=false 2023-07-20 02:55:43,340 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:43,371 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:43,446 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:43,479 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:43,500 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:43,500 INFO [M:0;jenkins-hbase4:45829] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45829,1689821736619; zookeeper connection closed. 2023-07-20 02:55:43,500 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:45829-0x10180a7ca410007, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:43,553 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:43,589 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:43,600 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:43,600 INFO [RS:3;jenkins-hbase4:46489] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46489,1689821736748; zookeeper connection closed. 2023-07-20 02:55:43,600 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:46489-0x10180a7ca41000b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:43,601 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@59bef6d9] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@59bef6d9 2023-07-20 02:55:43,658 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:43,696 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:43,700 INFO [jenkins-hbase4:46185Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 200, current progress: walGroup [jenkins-hbase4.apache.org%2C46185%2C1689821732333]: currently replicating from: hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822 at position: 22269 2023-07-20 02:55:43,700 INFO [jenkins-hbase4:39331Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 100, current progress: walGroup [jenkins-hbase4.apache.org%2C39331%2C1689821732285]: currently replicating from: hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822 at position: 10975 2023-07-20 02:55:43,764 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:43,800 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:43,800 INFO [RS:0;jenkins-hbase4:44133] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44133,1689821736661; zookeeper connection closed. 2023-07-20 02:55:43,801 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:44133-0x10180a7ca410008, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:43,802 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4d358804] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4d358804 2023-07-20 02:55:43,802 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:43,869 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:43,907 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:43,975 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 02:55:44,001 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:44,001 INFO [RS:1;jenkins-hbase4:40959] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40959,1689821736691; zookeeper connection closed. 2023-07-20 02:55:44,001 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:40959-0x10180a7ca410009, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:44,002 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@39a70d05] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@39a70d05 2023-07-20 02:55:44,003 INFO [Listener at localhost/46633] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-07-20 02:55:44,012 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-20 02:55:44,025 INFO [Listener at localhost/46633] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-20 02:55:44,025 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7f32269b to 127.0.0.1:55129 2023-07-20 02:55:44,025 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:44,025 DEBUG [Listener at localhost/46633] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-20 02:55:44,025 DEBUG [Listener at localhost/46633] util.JVMClusterUtil(257): Found active master hash=599010081, stopped=false 2023-07-20 02:55:44,025 INFO [Listener at localhost/46633] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,36235,1689821731026 2023-07-20 02:55:44,028 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-20 02:55:44,028 INFO [Listener at localhost/46633] procedure2.ProcedureExecutor(629): Stopping 2023-07-20 02:55:44,029 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:44,029 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3e74e971 to 127.0.0.1:55129 2023-07-20 02:55:44,028 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-20 02:55:44,029 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:44,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 02:55:44,028 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-20 02:55:44,030 INFO [Listener at localhost/46633] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39331,1689821732285' ***** 2023-07-20 02:55:44,031 INFO [Listener at localhost/46633] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 02:55:44,031 INFO [Listener at localhost/46633] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46185,1689821732333' ***** 2023-07-20 02:55:44,031 INFO [Listener at localhost/46633] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 02:55:44,031 INFO [RS:0;jenkins-hbase4:39331] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 02:55:44,031 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 02:55:44,031 INFO [RS:1;jenkins-hbase4:46185] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 02:55:44,031 INFO [RS:0;jenkins-hbase4:39331] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 02:55:44,033 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 02:55:44,033 INFO [RS:0;jenkins-hbase4:39331] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 02:55:44,033 INFO [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(3305): Received CLOSE for 300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:44,033 INFO [RS:1;jenkins-hbase4:46185] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 02:55:44,034 INFO [RS:1;jenkins-hbase4:46185] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 02:55:44,034 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(3305): Received CLOSE for 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:55:44,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 02:55:44,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 02:55:44,034 INFO [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:44,034 DEBUG [RS:0;jenkins-hbase4:39331] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x16f27e27 to 127.0.0.1:55129 2023-07-20 02:55:44,034 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(3305): Received CLOSE for 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:44,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 300903cec0fd231ca88313bcf649662a, disabling compactions & flushes 2023-07-20 02:55:44,035 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:44,034 DEBUG [RS:0;jenkins-hbase4:39331] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:44,036 DEBUG [RS:1;jenkins-hbase4:46185] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6ee0915f to 127.0.0.1:55129 2023-07-20 02:55:44,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:44,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1af5feb4e30a65bed26cf0457e90777d, disabling compactions & flushes 2023-07-20 02:55:44,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:44,036 DEBUG [RS:1;jenkins-hbase4:46185] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:44,036 INFO [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-20 02:55:44,036 INFO [RS:1;jenkins-hbase4:46185] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 02:55:44,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. after waiting 0 ms 2023-07-20 02:55:44,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:44,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:44,036 INFO [RS:1;jenkins-hbase4:46185] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 02:55:44,036 DEBUG [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1478): Online Regions={300903cec0fd231ca88313bcf649662a=t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.} 2023-07-20 02:55:44,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 300903cec0fd231ca88313bcf649662a 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-07-20 02:55:44,036 INFO [RS:1;jenkins-hbase4:46185] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 02:55:44,037 DEBUG [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1504): Waiting on 300903cec0fd231ca88313bcf649662a 2023-07-20 02:55:44,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:44,037 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-20 02:55:44,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. after waiting 0 ms 2023-07-20 02:55:44,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:44,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1af5feb4e30a65bed26cf0457e90777d 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-20 02:55:44,037 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-07-20 02:55:44,037 DEBUG [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1478): Online Regions={1af5feb4e30a65bed26cf0457e90777d=hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d., 392fe9dfffd6a3888f97c341c2952aeb=t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb., 1588230740=hbase:meta,,1.1588230740} 2023-07-20 02:55:44,038 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 02:55:44,038 DEBUG [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1504): Waiting on 1588230740, 1af5feb4e30a65bed26cf0457e90777d, 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:44,038 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 02:55:44,038 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 02:55:44,038 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 02:55:44,038 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 02:55:44,038 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-07-20 02:55:44,074 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/.tmp/info/da07c5a494c2484a8c01dd3b447a87ae 2023-07-20 02:55:44,075 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/.tmp/info/86eea25cb55d41a68aa2a9f148b9ec9d 2023-07-20 02:55:44,076 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:44,076 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:44,079 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/.tmp/cf1/b972b96393274eb18c7163b9cb64372d 2023-07-20 02:55:44,088 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-20 02:55:44,088 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-20 02:55:44,088 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-20 02:55:44,088 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/300903cec0fd231ca88313bcf649662a/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822, lastWalPosition=14562, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-07-20 02:55:44,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/.tmp/info/86eea25cb55d41a68aa2a9f148b9ec9d as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/info/86eea25cb55d41a68aa2a9f148b9ec9d 2023-07-20 02:55:44,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b972b96393274eb18c7163b9cb64372d 2023-07-20 02:55:44,098 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/info/86eea25cb55d41a68aa2a9f148b9ec9d, entries=2, sequenceid=6, filesize=4.8 K 2023-07-20 02:55:44,102 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 1af5feb4e30a65bed26cf0457e90777d in 65ms, sequenceid=6, compaction requested=false 2023-07-20 02:55:44,115 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/.tmp/rep_barrier/7c143364f9ad48829a6ce0d77b4e3124 2023-07-20 02:55:44,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-20 02:55:44,127 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/.tmp/norep/2efb46b211354194bf6677b74b8dba05 2023-07-20 02:55:44,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:44,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1af5feb4e30a65bed26cf0457e90777d: 2023-07-20 02:55:44,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:55:44,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 392fe9dfffd6a3888f97c341c2952aeb, disabling compactions & flushes 2023-07-20 02:55:44,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:44,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:44,130 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. after waiting 0 ms 2023-07-20 02:55:44,130 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:44,130 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 392fe9dfffd6a3888f97c341c2952aeb 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-07-20 02:55:44,131 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 29747 2023-07-20 02:55:44,131 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-20 02:55:44,131 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 29747 2023-07-20 02:55:44,131 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822, lastWalPosition=29747, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-07-20 02:55:44,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2efb46b211354194bf6677b74b8dba05 2023-07-20 02:55:44,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/.tmp/cf1/b972b96393274eb18c7163b9cb64372d as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/cf1/b972b96393274eb18c7163b9cb64372d 2023-07-20 02:55:44,151 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/.tmp/table/03dd25f3e2864c21837ddbdf1e79a2f0 2023-07-20 02:55:44,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b972b96393274eb18c7163b9cb64372d 2023-07-20 02:55:44,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/cf1/b972b96393274eb18c7163b9cb64372d, entries=100, sequenceid=106, filesize=8.2 K 2023-07-20 02:55:44,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/.tmp/norep/2efb46b211354194bf6677b74b8dba05 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/norep/2efb46b211354194bf6677b74b8dba05 2023-07-20 02:55:44,167 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/.tmp/info/da07c5a494c2484a8c01dd3b447a87ae as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/info/da07c5a494c2484a8c01dd3b447a87ae 2023-07-20 02:55:44,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/.tmp/cf1/29c24efb07ad45499008086172519b02 2023-07-20 02:55:44,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2efb46b211354194bf6677b74b8dba05 2023-07-20 02:55:44,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/norep/2efb46b211354194bf6677b74b8dba05, entries=51, sequenceid=106, filesize=6.5 K 2023-07-20 02:55:44,176 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/info/da07c5a494c2484a8c01dd3b447a87ae, entries=30, sequenceid=19, filesize=8.1 K 2023-07-20 02:55:44,176 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 29c24efb07ad45499008086172519b02 2023-07-20 02:55:44,177 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/.tmp/rep_barrier/7c143364f9ad48829a6ce0d77b4e3124 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/rep_barrier/7c143364f9ad48829a6ce0d77b4e3124 2023-07-20 02:55:44,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for 300903cec0fd231ca88313bcf649662a in 141ms, sequenceid=106, compaction requested=false 2023-07-20 02:55:44,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-07-20 02:55:44,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:44,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 300903cec0fd231ca88313bcf649662a: 2023-07-20 02:55:44,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:55:44,192 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/rep_barrier/7c143364f9ad48829a6ce0d77b4e3124, entries=2, sequenceid=19, filesize=4.9 K 2023-07-20 02:55:44,193 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/.tmp/table/03dd25f3e2864c21837ddbdf1e79a2f0 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/table/03dd25f3e2864c21837ddbdf1e79a2f0 2023-07-20 02:55:44,198 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-07-20 02:55:44,198 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 02:55:44,198 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-07-20 02:55:44,199 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/.tmp/norep/4a88f5547efe4f2eb883aa25fb8a0014 2023-07-20 02:55:44,204 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/table/03dd25f3e2864c21837ddbdf1e79a2f0, entries=6, sequenceid=19, filesize=4.8 K 2023-07-20 02:55:44,205 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 166ms, sequenceid=19, compaction requested=false 2023-07-20 02:55:44,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4a88f5547efe4f2eb883aa25fb8a0014 2023-07-20 02:55:44,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/.tmp/cf1/29c24efb07ad45499008086172519b02 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/cf1/29c24efb07ad45499008086172519b02 2023-07-20 02:55:44,219 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-20 02:55:44,219 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 02:55:44,220 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 02:55:44,220 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 02:55:44,220 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-20 02:55:44,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 29c24efb07ad45499008086172519b02 2023-07-20 02:55:44,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/cf1/29c24efb07ad45499008086172519b02, entries=200, sequenceid=206, filesize=11.9 K 2023-07-20 02:55:44,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/.tmp/norep/4a88f5547efe4f2eb883aa25fb8a0014 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/norep/4a88f5547efe4f2eb883aa25fb8a0014 2023-07-20 02:55:44,230 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4a88f5547efe4f2eb883aa25fb8a0014 2023-07-20 02:55:44,230 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/norep/4a88f5547efe4f2eb883aa25fb8a0014, entries=101, sequenceid=206, filesize=8.3 K 2023-07-20 02:55:44,231 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for 392fe9dfffd6a3888f97c341c2952aeb in 101ms, sequenceid=206, compaction requested=false 2023-07-20 02:55:44,237 INFO [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39331,1689821732285; all regions closed. 2023-07-20 02:55:44,238 DEBUG [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1504): Waiting on 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:55:44,243 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 30605 2023-07-20 02:55:44,244 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 02:55:44,244 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 30605 2023-07-20 02:55:44,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-07-20 02:55:44,246 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:44,246 DEBUG [RS:0;jenkins-hbase4:39331] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs 2023-07-20 02:55:44,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 392fe9dfffd6a3888f97c341c2952aeb: 2023-07-20 02:55:44,246 INFO [RS:0;jenkins-hbase4:39331] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39331%2C1689821732285:(num 1689821733822) 2023-07-20 02:55:44,246 DEBUG [RS:0;jenkins-hbase4:39331] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:44,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:55:44,246 INFO [RS:0;jenkins-hbase4:39331] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:44,246 INFO [RS:0;jenkins-hbase4:39331] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 02:55:44,246 INFO [RS:0;jenkins-hbase4:39331] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 02:55:44,246 INFO [RS:0;jenkins-hbase4:39331] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 02:55:44,246 INFO [RS:0;jenkins-hbase4:39331] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 02:55:44,246 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:55:44,247 INFO [RS:0;jenkins-hbase4:39331] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-20 02:55:44,299 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822 was moved to hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822 2023-07-20 02:55:44,350 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 30605 2023-07-20 02:55:44,438 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46185,1689821732333; all regions closed. 2023-07-20 02:55:44,442 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.meta.1689821734039.meta not finished, retry = 0 2023-07-20 02:55:44,455 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.ProtobufLogReader(420): EOF at position 30605 2023-07-20 02:55:44,546 DEBUG [RS:1;jenkins-hbase4:46185] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs 2023-07-20 02:55:44,546 INFO [RS:1;jenkins-hbase4:46185] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46185%2C1689821732333.meta:.meta(num 1689821734039) 2023-07-20 02:55:44,553 DEBUG [RS:1;jenkins-hbase4:46185] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs 2023-07-20 02:55:44,553 INFO [RS:1;jenkins-hbase4:46185] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46185%2C1689821732333:(num 1689821733822) 2023-07-20 02:55:44,553 DEBUG [RS:1;jenkins-hbase4:46185] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:44,553 INFO [RS:1;jenkins-hbase4:46185] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:55:44,553 INFO [RS:1;jenkins-hbase4:46185] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 02:55:44,553 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:55:44,554 INFO [RS:1;jenkins-hbase4:46185] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-20 02:55:44,557 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822 was moved to hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822 2023-07-20 02:55:44,602 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410017, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:44,602 DEBUG [RS:0;jenkins-hbase4:39331] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1cf903ed to 127.0.0.1:55129 2023-07-20 02:55:44,602 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410017, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:44,603 DEBUG [RS:0;jenkins-hbase4:39331] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:44,603 INFO [RS:0;jenkins-hbase4:39331] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1 terminated 2023-07-20 02:55:44,604 INFO [RS:0;jenkins-hbase4:39331] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39331 2023-07-20 02:55:44,608 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:44,608 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 02:55:44,608 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 02:55:44,608 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:44,608 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 02:55:44,610 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39331,1689821732285] 2023-07-20 02:55:44,610 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39331,1689821732285; numProcessing=1 2023-07-20 02:55:44,612 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,39331,1689821732285 already deleted, retry=false 2023-07-20 02:55:44,612 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39331,1689821732285 expired; onlineServers=1 2023-07-20 02:55:44,702 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410018, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:44,703 DEBUG [RS:1;jenkins-hbase4:46185] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5c078677 to 127.0.0.1:55129 2023-07-20 02:55:44,703 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410018, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:44,703 DEBUG [RS:1;jenkins-hbase4:46185] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:44,703 INFO [RS:1;jenkins-hbase4:46185] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1 terminated 2023-07-20 02:55:44,704 INFO [RS:1;jenkins-hbase4:46185] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46185 2023-07-20 02:55:44,706 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:55:44,706 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 02:55:44,708 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46185,1689821732333] 2023-07-20 02:55:44,708 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46185,1689821732333; numProcessing=2 2023-07-20 02:55:44,709 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,46185,1689821732333 already deleted, retry=false 2023-07-20 02:55:44,709 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46185,1689821732333 expired; onlineServers=0 2023-07-20 02:55:44,709 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36235,1689821731026' ***** 2023-07-20 02:55:44,709 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-20 02:55:44,710 DEBUG [M:0;jenkins-hbase4:36235] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6289211a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:55:44,710 INFO [M:0;jenkins-hbase4:36235] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36235,1689821731026 2023-07-20 02:55:44,711 INFO [M:0;jenkins-hbase4:36235] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36235,1689821731026; all regions closed. 2023-07-20 02:55:44,711 DEBUG [M:0;jenkins-hbase4:36235] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:44,711 DEBUG [M:0;jenkins-hbase4:36235] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-20 02:55:44,711 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-20 02:55:44,711 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821733512] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821733512,5,FailOnTimeoutGroup] 2023-07-20 02:55:44,711 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821733513] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821733513,5,FailOnTimeoutGroup] 2023-07-20 02:55:44,711 DEBUG [M:0;jenkins-hbase4:36235] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-20 02:55:44,713 INFO [M:0;jenkins-hbase4:36235] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-20 02:55:44,713 INFO [M:0;jenkins-hbase4:36235] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-20 02:55:44,713 INFO [M:0;jenkins-hbase4:36235] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-20 02:55:44,713 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-20 02:55:44,713 DEBUG [M:0;jenkins-hbase4:36235] master.HMaster(1512): Stopping service threads 2023-07-20 02:55:44,713 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:55:44,713 INFO [M:0;jenkins-hbase4:36235] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-20 02:55:44,714 ERROR [M:0;jenkins-hbase4:36235] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-20 02:55:44,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-07-20 02:55:44,714 INFO [M:0;jenkins-hbase4:36235] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-20 02:55:44,714 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-20 02:55:44,714 DEBUG [M:0;jenkins-hbase4:36235] zookeeper.ZKUtil(398): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-20 02:55:44,714 WARN [M:0;jenkins-hbase4:36235] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-20 02:55:44,714 INFO [M:0;jenkins-hbase4:36235] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-20 02:55:44,715 INFO [M:0;jenkins-hbase4:36235] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-20 02:55:44,716 DEBUG [M:0;jenkins-hbase4:36235] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 02:55:44,716 INFO [M:0;jenkins-hbase4:36235] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:44,716 DEBUG [M:0;jenkins-hbase4:36235] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:44,716 DEBUG [M:0;jenkins-hbase4:36235] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 02:55:44,716 DEBUG [M:0;jenkins-hbase4:36235] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:44,716 INFO [M:0;jenkins-hbase4:36235] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.04 KB heapSize=72.60 KB 2023-07-20 02:55:44,732 INFO [M:0;jenkins-hbase4:36235] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.04 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3bab9e7494f1457c839490c938450f35 2023-07-20 02:55:44,738 DEBUG [M:0;jenkins-hbase4:36235] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3bab9e7494f1457c839490c938450f35 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3bab9e7494f1457c839490c938450f35 2023-07-20 02:55:44,744 INFO [M:0;jenkins-hbase4:36235] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3bab9e7494f1457c839490c938450f35, entries=17, sequenceid=136, filesize=8.5 K 2023-07-20 02:55:44,745 INFO [M:0;jenkins-hbase4:36235] regionserver.HRegion(2948): Finished flush of dataSize ~61.04 KB/62508, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 29ms, sequenceid=136, compaction requested=false 2023-07-20 02:55:44,747 INFO [M:0;jenkins-hbase4:36235] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:55:44,748 DEBUG [M:0;jenkins-hbase4:36235] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:55:44,751 INFO [M:0;jenkins-hbase4:36235] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-20 02:55:44,751 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:55:44,752 INFO [M:0;jenkins-hbase4:36235] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36235 2023-07-20 02:55:44,756 DEBUG [M:0;jenkins-hbase4:36235] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,36235,1689821731026 already deleted, retry=false 2023-07-20 02:55:44,803 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:44,803 INFO [RS:0;jenkins-hbase4:39331] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39331,1689821732285; zookeeper connection closed. 2023-07-20 02:55:44,803 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:39331-0x10180a7ca410001, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:44,803 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@785ca876] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@785ca876 2023-07-20 02:55:44,946 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 02:55:45,028 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:45,028 INFO [M:0;jenkins-hbase4:36235] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36235,1689821731026; zookeeper connection closed. 2023-07-20 02:55:45,029 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): master:36235-0x10180a7ca410000, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:45,129 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:45,129 INFO [RS:1;jenkins-hbase4:46185] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46185,1689821732333; zookeeper connection closed. 2023-07-20 02:55:45,129 DEBUG [Listener at localhost/35915-EventThread] zookeeper.ZKWatcher(600): regionserver:46185-0x10180a7ca410002, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:55:45,129 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7b019929] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7b019929 2023-07-20 02:55:45,129 INFO [Listener at localhost/46633] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-07-20 02:55:45,131 INFO [Listener at localhost/46633] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 02:55:45,132 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:45,132 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:45,132 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:55:45,132 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:45,132 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:55:45,132 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:55:45,133 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40859 2023-07-20 02:55:45,134 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:45,135 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:45,136 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=master:40859 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:45,140 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:408590x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:45,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:40859-0x10180a7ca41001d connected 2023-07-20 02:55:45,144 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:55:45,145 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:45,145 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 02:55:45,150 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40859 2023-07-20 02:55:45,150 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40859 2023-07-20 02:55:45,150 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40859 2023-07-20 02:55:45,151 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40859 2023-07-20 02:55:45,151 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40859 2023-07-20 02:55:45,152 INFO [Listener at localhost/46633] master.HMaster(444): hbase.rootdir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16, hbase.cluster.distributed=false 2023-07-20 02:55:45,155 DEBUG [pool-442-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-20 02:55:45,168 INFO [Listener at localhost/46633] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 02:55:45,168 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:45,168 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:45,168 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:55:45,168 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:55:45,168 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:55:45,168 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:55:45,169 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38383 2023-07-20 02:55:45,169 INFO [Listener at localhost/46633] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 02:55:45,172 DEBUG [Listener at localhost/46633] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 02:55:45,172 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:45,173 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:45,174 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38383 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:45,178 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:383830x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:45,179 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:383830x0, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:55:45,180 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:383830x0, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:55:45,180 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:383830x0, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 02:55:45,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38383-0x10180a7ca41001e connected 2023-07-20 02:55:45,187 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38383 2023-07-20 02:55:45,188 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38383 2023-07-20 02:55:45,189 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38383 2023-07-20 02:55:45,190 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38383 2023-07-20 02:55:45,190 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38383 2023-07-20 02:55:45,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,194 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 02:55:45,195 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,196 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 02:55:45,196 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38383-0x10180a7ca41001e, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 02:55:45,196 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:45,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 02:55:45,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 02:55:45,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,40859,1689821745131 from backup master directory 2023-07-20 02:55:45,200 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,200 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 02:55:45,200 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:55:45,200 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:55:45,245 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x520f6998 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:45,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11993146, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:45,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 02:55:45,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-20 02:55:45,254 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:45,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,45829,1689821736619 to hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,45829,1689821736619-dead as it is dead 2023-07-20 02:55:45,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,45829,1689821736619-dead/jenkins-hbase4.apache.org%2C45829%2C1689821736619.1689821736886 2023-07-20 02:55:45,268 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,45829,1689821736619-dead/jenkins-hbase4.apache.org%2C45829%2C1689821736619.1689821736886 after 4ms 2023-07-20 02:55:45,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,45829,1689821736619-dead/jenkins-hbase4.apache.org%2C45829%2C1689821736619.1689821736886 to hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45829%2C1689821736619.1689821736886 2023-07-20 02:55:45,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,45829,1689821736619-dead 2023-07-20 02:55:45,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,272 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40859%2C1689821745131, suffix=, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,40859,1689821745131, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/oldWALs, maxLogs=10 2023-07-20 02:55:45,288 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK] 2023-07-20 02:55:45,291 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK] 2023-07-20 02:55:45,289 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK] 2023-07-20 02:55:45,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,40859,1689821745131/jenkins-hbase4.apache.org%2C40859%2C1689821745131.1689821745272 2023-07-20 02:55:45,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK], DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK], DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK]] 2023-07-20 02:55:45,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:45,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:45,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:45,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:45,297 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:45,298 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-20 02:55:45,298 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-20 02:55:45,309 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7538bca05e2a4ba7834e6fa33e0cf942 2023-07-20 02:55:45,309 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:45,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-20 02:55:45,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45829%2C1689821736619.1689821736886 2023-07-20 02:55:45,317 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45829%2C1689821736619.1689821736886 2023-07-20 02:55:45,318 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45829%2C1689821736619.1689821736886 2023-07-20 02:55:45,322 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:55:45,324 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-07-20 02:55:45,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62267430, jitterRate=-0.07214298844337463}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 02:55:45,325 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:55:45,326 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-20 02:55:45,327 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-20 02:55:45,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-20 02:55:45,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-20 02:55:45,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-20 02:55:45,346 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-20 02:55:45,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-20 02:55:45,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-20 02:55:45,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-20 02:55:45,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-20 02:55:45,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-20 02:55:45,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 20 msec 2023-07-20 02:55:45,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-20 02:55:45,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-20 02:55:45,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46489,1689821736748, table=hbase:meta, region=1588230740 2023-07-20 02:55:45,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-20 02:55:45,355 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46489,1689821736748 already deleted, retry=false 2023-07-20 02:55:45,355 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46489,1689821736748 on jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46489,1689821736748, splitWal=true, meta=true 2023-07-20 02:55:45,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,46489,1689821736748 (carryingMeta=true) jenkins-hbase4.apache.org,46489,1689821736748/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@712b2217[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 02:55:45,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,40959,1689821736691 already deleted, retry=false 2023-07-20 02:55:45,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,40959,1689821736691 on jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,367 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,40959,1689821736691, splitWal=true, meta=false 2023-07-20 02:55:45,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase4.apache.org,40959,1689821736691 (carryingMeta=false) jenkins-hbase4.apache.org,40959,1689821736691/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@3f84f155[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 02:55:45,369 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,38819,1689821736718 already deleted, retry=false 2023-07-20 02:55:45,369 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,38819,1689821736718 on jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,370 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,38819,1689821736718, splitWal=true, meta=false 2023-07-20 02:55:45,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase4.apache.org,38819,1689821736718 (carryingMeta=false) jenkins-hbase4.apache.org,38819,1689821736718/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@17175728[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 02:55:45,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,44133,1689821736661 already deleted, retry=false 2023-07-20 02:55:45,372 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,44133,1689821736661 on jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,373 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,44133,1689821736661, splitWal=true, meta=false 2023-07-20 02:55:45,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,44133,1689821736661 (carryingMeta=false) jenkins-hbase4.apache.org,44133,1689821736661/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@6ef91f52[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 02:55:45,384 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-20 02:55:45,384 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-20 02:55:45,384 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-20 02:55:45,384 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-20 02:55:45,385 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-20 02:55:45,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-20 02:55:45,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-20 02:55:45,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-20 02:55:45,389 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38383-0x10180a7ca41001e, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 02:55:45,389 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 02:55:45,389 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:55:45,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,40859,1689821745131, sessionid=0x10180a7ca41001d, setting cluster-up flag (Was=false) 2023-07-20 02:55:45,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-20 02:55:45,396 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-20 02:55:45,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:55:45,401 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.hbase-snapshot/.tmp 2023-07-20 02:55:45,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:45,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:45,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:45,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:55:45,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-20 02:55:45,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:55:45,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689821775408 2023-07-20 02:55:45,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-20 02:55:45,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-20 02:55:45,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-20 02:55:45,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-20 02:55:45,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-20 02:55:45,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-20 02:55:45,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:45,414 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38819,1689821736718; numProcessing=1 2023-07-20 02:55:45,414 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46489,1689821736748; numProcessing=2 2023-07-20 02:55:45,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-20 02:55:45,414 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,44133,1689821736661; numProcessing=3 2023-07-20 02:55:45,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-20 02:55:45,415 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40959,1689821736691; numProcessing=4 2023-07-20 02:55:45,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-20 02:55:45,415 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,40959,1689821736691, splitWal=true, meta=false 2023-07-20 02:55:45,415 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38819,1689821736718, splitWal=true, meta=false 2023-07-20 02:55:45,415 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46489,1689821736748, splitWal=true, meta=true 2023-07-20 02:55:45,415 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,44133,1689821736661, splitWal=true, meta=false 2023-07-20 02:55:45,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-20 02:55:45,416 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-20 02:55:45,416 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821745416,5,FailOnTimeoutGroup] 2023-07-20 02:55:45,416 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46489,1689821736748, splitWal=true, meta=true, isMeta: true 2023-07-20 02:55:45,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821745416,5,FailOnTimeoutGroup] 2023-07-20 02:55:45,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:45,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-20 02:55:45,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:45,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:45,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689821745419, completionTime=-1 2023-07-20 02:55:45,420 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-20 02:55:45,420 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 02:55:45,421 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,46489,1689821736748-splitting 2023-07-20 02:55:45,422 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,46489,1689821736748-splitting dir is empty, no logs to split. 2023-07-20 02:55:45,422 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46489,1689821736748 WAL count=0, meta=true 2023-07-20 02:55:45,425 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,46489,1689821736748-splitting dir is empty, no logs to split. 2023-07-20 02:55:45,426 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46489,1689821736748 WAL count=0, meta=true 2023-07-20 02:55:45,426 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46489,1689821736748 WAL splitting is done? wals=0, meta=true 2023-07-20 02:55:45,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-20 02:55:45,428 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-20 02:55:45,429 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:55:45,495 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(951): ClusterId : 69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:55:45,495 DEBUG [RS:0;jenkins-hbase4:38383] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 02:55:45,497 DEBUG [RS:0;jenkins-hbase4:38383] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 02:55:45,497 DEBUG [RS:0;jenkins-hbase4:38383] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 02:55:45,499 DEBUG [RS:0;jenkins-hbase4:38383] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 02:55:45,501 DEBUG [RS:0;jenkins-hbase4:38383] zookeeper.ReadOnlyZKClient(139): Connect 0x600bf1f0 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:45,505 DEBUG [RS:0;jenkins-hbase4:38383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6ecc0381, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:45,505 DEBUG [RS:0;jenkins-hbase4:38383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6af39a80, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:55:45,514 DEBUG [RS:0;jenkins-hbase4:38383] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:38383 2023-07-20 02:55:45,514 INFO [RS:0;jenkins-hbase4:38383] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 02:55:45,514 INFO [RS:0;jenkins-hbase4:38383] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 02:55:45,514 DEBUG [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 02:55:45,515 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,40859,1689821745131 with isa=jenkins-hbase4.apache.org/172.31.14.131:38383, startcode=1689821745167 2023-07-20 02:55:45,515 DEBUG [RS:0;jenkins-hbase4:38383] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 02:55:45,517 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57693, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 02:55:45,519 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40859] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:45,519 DEBUG [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16 2023-07-20 02:55:45,519 DEBUG [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40517 2023-07-20 02:55:45,519 DEBUG [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 02:55:45,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=100ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 02:55:45,521 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:55:45,521 DEBUG [RS:0;jenkins-hbase4:38383] zookeeper.ZKUtil(162): regionserver:38383-0x10180a7ca41001e, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:45,522 WARN [RS:0;jenkins-hbase4:38383] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:55:45,522 INFO [RS:0;jenkins-hbase4:38383] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:45,522 DEBUG [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:45,522 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38383,1689821745167] 2023-07-20 02:55:45,527 DEBUG [RS:0;jenkins-hbase4:38383] zookeeper.ZKUtil(162): regionserver:38383-0x10180a7ca41001e, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:45,528 DEBUG [RS:0;jenkins-hbase4:38383] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:55:45,528 INFO [RS:0;jenkins-hbase4:38383] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 02:55:45,529 INFO [RS:0;jenkins-hbase4:38383] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 02:55:45,529 INFO [RS:0;jenkins-hbase4:38383] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 02:55:45,530 INFO [RS:0;jenkins-hbase4:38383] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:45,530 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 02:55:45,531 INFO [RS:0;jenkins-hbase4:38383] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:45,531 DEBUG [RS:0;jenkins-hbase4:38383] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,532 DEBUG [RS:0;jenkins-hbase4:38383] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,532 DEBUG [RS:0;jenkins-hbase4:38383] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,532 DEBUG [RS:0;jenkins-hbase4:38383] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,532 DEBUG [RS:0;jenkins-hbase4:38383] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,532 DEBUG [RS:0;jenkins-hbase4:38383] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:55:45,532 DEBUG [RS:0;jenkins-hbase4:38383] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,532 DEBUG [RS:0;jenkins-hbase4:38383] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,532 DEBUG [RS:0;jenkins-hbase4:38383] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,532 DEBUG [RS:0;jenkins-hbase4:38383] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:55:45,534 INFO [RS:0;jenkins-hbase4:38383] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:45,535 INFO [RS:0;jenkins-hbase4:38383] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:45,535 INFO [RS:0;jenkins-hbase4:38383] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:45,546 INFO [RS:0;jenkins-hbase4:38383] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 02:55:45,546 INFO [RS:0;jenkins-hbase4:38383] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38383,1689821745167-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:45,559 INFO [RS:0;jenkins-hbase4:38383] regionserver.Replication(203): jenkins-hbase4.apache.org,38383,1689821745167 started 2023-07-20 02:55:45,559 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38383,1689821745167, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38383, sessionid=0x10180a7ca41001e 2023-07-20 02:55:45,560 DEBUG [RS:0;jenkins-hbase4:38383] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 02:55:45,560 DEBUG [RS:0;jenkins-hbase4:38383] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:45,560 DEBUG [RS:0;jenkins-hbase4:38383] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38383,1689821745167' 2023-07-20 02:55:45,560 DEBUG [RS:0;jenkins-hbase4:38383] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 02:55:45,560 DEBUG [RS:0;jenkins-hbase4:38383] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 02:55:45,560 DEBUG [RS:0;jenkins-hbase4:38383] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 02:55:45,561 DEBUG [RS:0;jenkins-hbase4:38383] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 02:55:45,561 DEBUG [RS:0;jenkins-hbase4:38383] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:45,561 DEBUG [RS:0;jenkins-hbase4:38383] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38383,1689821745167' 2023-07-20 02:55:45,561 DEBUG [RS:0;jenkins-hbase4:38383] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 02:55:45,561 DEBUG [RS:0;jenkins-hbase4:38383] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 02:55:45,561 DEBUG [RS:0;jenkins-hbase4:38383] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 02:55:45,561 INFO [RS:0;jenkins-hbase4:38383] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 02:55:45,561 INFO [RS:0;jenkins-hbase4:38383] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 02:55:45,579 DEBUG [jenkins-hbase4:40859] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 02:55:45,581 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38383,1689821745167, state=OPENING 2023-07-20 02:55:45,584 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-20 02:55:45,584 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 02:55:45,584 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38383,1689821745167}] 2023-07-20 02:55:45,664 INFO [RS:0;jenkins-hbase4:38383] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38383%2C1689821745167, suffix=, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs, maxLogs=10 2023-07-20 02:55:45,683 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK] 2023-07-20 02:55:45,683 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK] 2023-07-20 02:55:45,683 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK] 2023-07-20 02:55:45,685 INFO [RS:0;jenkins-hbase4:38383] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167/jenkins-hbase4.apache.org%2C38383%2C1689821745167.1689821745665 2023-07-20 02:55:45,686 DEBUG [RS:0;jenkins-hbase4:38383] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK], DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK], DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK]] 2023-07-20 02:55:45,738 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:45,738 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:45,740 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58584, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:55:45,744 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-20 02:55:45,744 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:45,745 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38383%2C1689821745167.meta, suffix=.meta, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs, maxLogs=10 2023-07-20 02:55:45,760 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK] 2023-07-20 02:55:45,760 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK] 2023-07-20 02:55:45,760 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK] 2023-07-20 02:55:45,762 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167/jenkins-hbase4.apache.org%2C38383%2C1689821745167.meta.1689821745746.meta 2023-07-20 02:55:45,762 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK], DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK], DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK]] 2023-07-20 02:55:45,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:45,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-20 02:55:45,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-20 02:55:45,764 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-20 02:55:45,764 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-20 02:55:45,764 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:45,765 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-20 02:55:45,765 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-20 02:55:45,766 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 02:55:45,767 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info 2023-07-20 02:55:45,767 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info 2023-07-20 02:55:45,768 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 02:55:45,774 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info/441251dfb1734be1aa02887ae7be7817 2023-07-20 02:55:45,774 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:45,775 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 02:55:45,775 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:55:45,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:55:45,776 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 02:55:45,776 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:45,777 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 02:55:45,777 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table 2023-07-20 02:55:45,777 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table 2023-07-20 02:55:45,778 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 02:55:45,784 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table/838ab0139f504a0b99265e300d89842c 2023-07-20 02:55:45,784 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:45,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740 2023-07-20 02:55:45,786 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740 2023-07-20 02:55:45,788 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 02:55:45,790 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 02:55:45,791 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64657039, jitterRate=-0.03653503954410553}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 02:55:45,791 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 02:55:45,791 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1689821745738 2023-07-20 02:55:45,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-20 02:55:45,796 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-20 02:55:45,796 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38383,1689821745167, state=OPEN 2023-07-20 02:55:45,798 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-20 02:55:45,798 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 02:55:45,800 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-20 02:55:45,800 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38383,1689821745167 in 214 msec 2023-07-20 02:55:45,802 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=15 2023-07-20 02:55:45,802 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 373 msec 2023-07-20 02:55:47,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1602ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-20 02:55:48,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3105ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-20 02:55:49,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-20 02:55:49,928 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-20 02:55:49,931 DEBUG [hconnection-0x484d27f6-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:49,933 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58590, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:49,942 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=956fe6fd22cc055b7d5c09ea7e2fd4a7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44133,1689821736661, regionLocation=jenkins-hbase4.apache.org,44133,1689821736661, openSeqNum=2 2023-07-20 02:55:49,942 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=80b6625deaec89ca31a4c1b65e4a6544, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44133,1689821736661, regionLocation=jenkins-hbase4.apache.org,44133,1689821736661, openSeqNum=2 2023-07-20 02:55:49,942 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=6908f43230333d97246933b2af61e211, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40959,1689821736691, regionLocation=jenkins-hbase4.apache.org,40959,1689821736691, openSeqNum=2 2023-07-20 02:55:49,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-20 02:55:49,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689821809943 2023-07-20 02:55:49,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689821869943 2023-07-20 02:55:49,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 15 msec 2023-07-20 02:55:49,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40859,1689821745131-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:49,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40859,1689821745131-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:49,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40859,1689821745131-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:49,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:40859, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:49,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-20 02:55:49,971 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,38819,1689821736718 had 0 regions 2023-07-20 02:55:49,971 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46489,1689821736748 had 1 regions 2023-07-20 02:55:49,971 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,40959,1689821736691 had 1 regions 2023-07-20 02:55:49,971 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,44133,1689821736661 had 2 regions 2023-07-20 02:55:49,971 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. is NOT online; state={956fe6fd22cc055b7d5c09ea7e2fd4a7 state=OPEN, ts=1689821749942, server=jenkins-hbase4.apache.org,44133,1689821736661}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-20 02:55:49,972 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38819,1689821736718, splitWal=true, meta=false, isMeta: false 2023-07-20 02:55:49,973 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,40959,1689821736691, splitWal=true, meta=false, isMeta: false 2023-07-20 02:55:49,973 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46489,1689821736748, splitWal=true, meta=true, isMeta: false 2023-07-20 02:55:49,973 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,44133,1689821736661, splitWal=true, meta=false, isMeta: false 2023-07-20 02:55:49,974 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38819,1689821736718-splitting 2023-07-20 02:55:49,976 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38819,1689821736718-splitting dir is empty, no logs to split. 2023-07-20 02:55:49,976 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,40959,1689821736691-splitting 2023-07-20 02:55:49,976 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,38819,1689821736718 WAL count=0, meta=false 2023-07-20 02:55:49,978 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,40959,1689821736691-splitting dir is empty, no logs to split. 2023-07-20 02:55:49,978 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,40959,1689821736691 WAL count=0, meta=false 2023-07-20 02:55:49,978 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,44133,1689821736661/hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7., unknown_server=jenkins-hbase4.apache.org,44133,1689821736661/t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544., unknown_server=jenkins-hbase4.apache.org,40959,1689821736691/t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:49,979 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,44133,1689821736661-splitting 2023-07-20 02:55:49,979 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,46489,1689821736748-splitting dir is empty, no logs to split. 2023-07-20 02:55:49,979 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,46489,1689821736748 WAL count=0, meta=false 2023-07-20 02:55:49,980 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,44133,1689821736661-splitting dir is empty, no logs to split. 2023-07-20 02:55:49,980 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,44133,1689821736661 WAL count=0, meta=false 2023-07-20 02:55:49,981 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38819,1689821736718-splitting dir is empty, no logs to split. 2023-07-20 02:55:49,981 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,38819,1689821736718 WAL count=0, meta=false 2023-07-20 02:55:49,981 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,38819,1689821736718 WAL splitting is done? wals=0, meta=false 2023-07-20 02:55:49,982 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,40959,1689821736691-splitting dir is empty, no logs to split. 2023-07-20 02:55:49,982 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,40959,1689821736691 WAL count=0, meta=false 2023-07-20 02:55:49,982 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,40959,1689821736691 WAL splitting is done? wals=0, meta=false 2023-07-20 02:55:49,983 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,46489,1689821736748-splitting dir is empty, no logs to split. 2023-07-20 02:55:49,983 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,46489,1689821736748 WAL count=0, meta=false 2023-07-20 02:55:49,983 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46489,1689821736748 WAL splitting is done? wals=0, meta=false 2023-07-20 02:55:49,985 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,44133,1689821736661-splitting dir is empty, no logs to split. 2023-07-20 02:55:49,985 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,44133,1689821736661 WAL count=0, meta=false 2023-07-20 02:55:49,985 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,44133,1689821736661 WAL splitting is done? wals=0, meta=false 2023-07-20 02:55:49,985 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,38819,1689821736718 failed, ignore...File hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38819,1689821736718-splitting does not exist. 2023-07-20 02:55:49,985 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,40959,1689821736691 failed, ignore...File hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,40959,1689821736691-splitting does not exist. 2023-07-20 02:55:49,987 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46489,1689821736748 after splitting done 2023-07-20 02:55:49,987 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46489,1689821736748 from processing; numProcessing=3 2023-07-20 02:55:49,987 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN}] 2023-07-20 02:55:49,987 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,38819,1689821736718 after splitting done 2023-07-20 02:55:49,987 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,38819,1689821736718 from processing; numProcessing=2 2023-07-20 02:55:49,988 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,44133,1689821736661 failed, ignore...File hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,44133,1689821736661-splitting does not exist. 2023-07-20 02:55:49,989 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN 2023-07-20 02:55:49,990 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN}, {pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN}] 2023-07-20 02:55:49,990 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46489,1689821736748, splitWal=true, meta=true in 4.6310 sec 2023-07-20 02:55:49,990 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:55:49,990 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,38819,1689821736718, splitWal=true, meta=false in 4.6180 sec 2023-07-20 02:55:49,991 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN 2023-07-20 02:55:49,991 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN 2023-07-20 02:55:49,993 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:55:49,993 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:55:49,993 DEBUG [jenkins-hbase4:40859] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 02:55:49,996 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=6908f43230333d97246933b2af61e211, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:49,996 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821749996"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821749996"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821749996"}]},"ts":"1689821749996"} 2023-07-20 02:55:49,996 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=956fe6fd22cc055b7d5c09ea7e2fd4a7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:49,996 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=80b6625deaec89ca31a4c1b65e4a6544, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:49,997 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821749996"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821749996"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821749996"}]},"ts":"1689821749996"} 2023-07-20 02:55:49,997 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821749996"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821749996"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821749996"}]},"ts":"1689821749996"} 2023-07-20 02:55:49,999 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=21, state=RUNNABLE; OpenRegionProcedure 6908f43230333d97246933b2af61e211, server=jenkins-hbase4.apache.org,38383,1689821745167}] 2023-07-20 02:55:50,000 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=23, state=RUNNABLE; OpenRegionProcedure 956fe6fd22cc055b7d5c09ea7e2fd4a7, server=jenkins-hbase4.apache.org,38383,1689821745167}] 2023-07-20 02:55:50,001 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=22, state=RUNNABLE; OpenRegionProcedure 80b6625deaec89ca31a4c1b65e4a6544, server=jenkins-hbase4.apache.org,38383,1689821745167}] 2023-07-20 02:55:50,107 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 02:55:50,173 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:50,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 956fe6fd22cc055b7d5c09ea7e2fd4a7, NAME => 'hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:50,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:50,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:50,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:50,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:50,176 INFO [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:50,177 DEBUG [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/info 2023-07-20 02:55:50,177 DEBUG [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/info 2023-07-20 02:55:50,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:50,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 80b6625deaec89ca31a4c1b65e4a6544, NAME => 't1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:50,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:50,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:50,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:50,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:50,179 INFO [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 956fe6fd22cc055b7d5c09ea7e2fd4a7 columnFamilyName info 2023-07-20 02:55:50,181 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:50,183 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1 2023-07-20 02:55:50,183 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1 2023-07-20 02:55:50,184 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80b6625deaec89ca31a4c1b65e4a6544 columnFamilyName cf1 2023-07-20 02:55:50,189 DEBUG [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/info/ba087245ad7f4c18b071b799180f415c 2023-07-20 02:55:50,189 INFO [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] regionserver.HStore(310): Store=956fe6fd22cc055b7d5c09ea7e2fd4a7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:50,190 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:50,191 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:50,192 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1/db3cbecb4ef74734b4315fa2f2bf0702 2023-07-20 02:55:50,192 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(310): Store=80b6625deaec89ca31a4c1b65e4a6544/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:50,192 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:50,193 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/norep 2023-07-20 02:55:50,193 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/norep 2023-07-20 02:55:50,194 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80b6625deaec89ca31a4c1b65e4a6544 columnFamilyName norep 2023-07-20 02:55:50,194 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(310): Store=80b6625deaec89ca31a4c1b65e4a6544/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:50,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:50,195 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:55:50,196 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 956fe6fd22cc055b7d5c09ea7e2fd4a7; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69650235, jitterRate=0.03786937892436981}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 02:55:50,196 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 956fe6fd22cc055b7d5c09ea7e2fd4a7: 2023-07-20 02:55:50,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:50,197 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7., pid=25, masterSystemTime=1689821750154 2023-07-20 02:55:50,200 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:50,200 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:55:50,201 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=956fe6fd22cc055b7d5c09ea7e2fd4a7, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:50,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 02:55:50,201 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821750201"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821750201"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821750201"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821750201"}]},"ts":"1689821750201"} 2023-07-20 02:55:50,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:55:50,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 80b6625deaec89ca31a4c1b65e4a6544; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63644985, jitterRate=-0.051615819334983826}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 02:55:50,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 80b6625deaec89ca31a4c1b65e4a6544: 2023-07-20 02:55:50,205 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=23 2023-07-20 02:55:50,205 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=23, state=SUCCESS; OpenRegionProcedure 956fe6fd22cc055b7d5c09ea7e2fd4a7, server=jenkins-hbase4.apache.org,38383,1689821745167 in 203 msec 2023-07-20 02:55:50,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544., pid=26, masterSystemTime=1689821750154 2023-07-20 02:55:50,207 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN in 215 msec 2023-07-20 02:55:50,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:50,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:55:50,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:50,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6908f43230333d97246933b2af61e211, NAME => 't2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:55:50,208 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=80b6625deaec89ca31a4c1b65e4a6544, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:50,208 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821750208"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821750208"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821750208"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821750208"}]},"ts":"1689821750208"} 2023-07-20 02:55:50,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 6908f43230333d97246933b2af61e211 2023-07-20 02:55:50,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:55:50,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6908f43230333d97246933b2af61e211 2023-07-20 02:55:50,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6908f43230333d97246933b2af61e211 2023-07-20 02:55:50,210 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6908f43230333d97246933b2af61e211 2023-07-20 02:55:50,211 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1 2023-07-20 02:55:50,211 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1 2023-07-20 02:55:50,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=22 2023-07-20 02:55:50,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=22, state=SUCCESS; OpenRegionProcedure 80b6625deaec89ca31a4c1b65e4a6544, server=jenkins-hbase4.apache.org,38383,1689821745167 in 209 msec 2023-07-20 02:55:50,212 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6908f43230333d97246933b2af61e211 columnFamilyName cf1 2023-07-20 02:55:50,214 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=18 2023-07-20 02:55:50,214 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,44133,1689821736661 after splitting done 2023-07-20 02:55:50,214 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN in 222 msec 2023-07-20 02:55:50,215 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,44133,1689821736661 from processing; numProcessing=1 2023-07-20 02:55:50,217 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,44133,1689821736661, splitWal=true, meta=false in 4.8430 sec 2023-07-20 02:55:50,219 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1/a2587aab5db342319f7ea6dadbb7933e 2023-07-20 02:55:50,220 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(310): Store=6908f43230333d97246933b2af61e211/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:50,220 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 6908f43230333d97246933b2af61e211 2023-07-20 02:55:50,221 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/norep 2023-07-20 02:55:50,221 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/norep 2023-07-20 02:55:50,222 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6908f43230333d97246933b2af61e211 columnFamilyName norep 2023-07-20 02:55:50,222 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(310): Store=6908f43230333d97246933b2af61e211/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:55:50,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211 2023-07-20 02:55:50,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211 2023-07-20 02:55:50,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 02:55:50,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6908f43230333d97246933b2af61e211 2023-07-20 02:55:50,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6908f43230333d97246933b2af61e211; next sequenceid=12; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67181064, jitterRate=0.0010758638381958008}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 02:55:50,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6908f43230333d97246933b2af61e211: 2023-07-20 02:55:50,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211., pid=24, masterSystemTime=1689821750154 2023-07-20 02:55:50,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:50,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:55:50,231 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=6908f43230333d97246933b2af61e211, regionState=OPEN, openSeqNum=12, regionLocation=jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:55:50,231 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821750231"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821750231"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821750231"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821750231"}]},"ts":"1689821750231"} 2023-07-20 02:55:50,234 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=21 2023-07-20 02:55:50,234 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=21, state=SUCCESS; OpenRegionProcedure 6908f43230333d97246933b2af61e211, server=jenkins-hbase4.apache.org,38383,1689821745167 in 234 msec 2023-07-20 02:55:50,236 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=16 2023-07-20 02:55:50,236 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,40959,1689821736691 after splitting done 2023-07-20 02:55:50,236 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN in 248 msec 2023-07-20 02:55:50,236 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,40959,1689821736691 from processing; numProcessing=0 2023-07-20 02:55:50,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,40959,1689821736691, splitWal=true, meta=false in 4.8690 sec 2023-07-20 02:55:50,559 INFO [jenkins-hbase4:38383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:55:50,972 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-20 02:55:50,986 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-20 02:55:50,988 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-20 02:55:50,988 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.788sec 2023-07-20 02:55:50,988 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-20 02:55:50,988 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-20 02:55:50,988 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-20 02:55:50,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40859,1689821745131-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-20 02:55:50,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40859,1689821745131-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-20 02:55:50,992 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-20 02:55:50,999 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(139): Connect 0x53ea5f42 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:51,004 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@255702b4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:51,005 DEBUG [hconnection-0x21ba383f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:51,007 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40926, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:51,010 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-20 02:55:51,010 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x53ea5f42 to 127.0.0.1:55129 2023-07-20 02:55:51,010 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:55:51,013 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:40859 after: jenkins-hbase4.apache.org:40859 2023-07-20 02:55:51,013 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(139): Connect 0x5dcc4257 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:51,022 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d47f211, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:51,524 DEBUG [hconnection-0x7b78835a-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:51,526 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40942, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:55:51,528 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-20 02:55:51,529 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-20 02:55:51,534 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-20 02:55:51,535 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-20 02:55:51,552 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689821751552 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:51,555 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): syncupReplication16898217515520x0, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:51,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689821751552-0x10180a7ca410023 connected Start Replication Server start 2023-07-20 02:55:51,558 INFO [Listener at localhost/46633] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:55:51,571 INFO [Listener at localhost/46633] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 02:55:51,572 DEBUG [Listener at localhost/46633] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:55:51,583 INFO [Listener at localhost/46633] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-20 02:55:51,587 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,39331,1689821732285, jenkins-hbase4.apache.org,46185,1689821732333] other RSs: [] 2023-07-20 02:55:51,607 DEBUG [Listener at localhost/46633.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x19e03ec9 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:51,614 DEBUG [Listener at localhost/46633.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@13bb60e6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:51,615 INFO [Listener at localhost/46633.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:51,618 DEBUG [Listener at localhost/46633.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:51,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10180a7ca410025 connected 2023-07-20 02:55:51,619 INFO [Listener at localhost/46633.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=bbd9fa59-2213-4359-9606-c616f1cd40eb to cluster=69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:55:54,590 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,39331,1689821732285/1's WALs to 1689821751558.syncuptool.replication.org,1234,1 2023-07-20 02:55:54,593 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822 with data PBUF\x08\xDFU 2023-07-20 02:55:54,595 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,39331,1689821732285/1's WALs to 1689821751558.syncuptool.replication.org,1234,1 2023-07-20 02:55:55,559 INFO [jenkins-hbase4:38383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:55:55,678 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 02:55:56,598 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,39331,1689821732285] 2023-07-20 02:55:56,598 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,39331,1689821732285] 2023-07-20 02:55:56,613 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,39331,1689821732285] 2023-07-20 02:55:56,613 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,39331,1689821732285, ReplicationSource: 1, currentBandwidth=0 2023-07-20 02:55:56,613 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:56,634 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285] zookeeper.ReadOnlyZKClient(139): Connect 0x4c3a163b to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:56,641 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@40b88fe4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:56,641 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:55:56,645 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:55:56,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10180a7ca410027 connected 2023-07-20 02:55:56,646 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,39331,1689821732285 (queues=1) is replicating from cluster=bbd9fa59-2213-4359-9606-c616f1cd40eb to cluster=69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:55:56,646 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C39331%2C1689821732285 2023-07-20 02:55:56,650 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,39331,1689821732285, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 02:55:56,651 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C39331%2C1689821732285 2023-07-20 02:55:56,658 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822 2023-07-20 02:55:56,658 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-20 02:55:56,658 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-20 02:55:56,659 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,39331,1689821732285 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/300903cec0fd231ca88313bcf649662a/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs/jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822, lastWalPosition=15173, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-07-20 02:55:56,661 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:55:56,663 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40956, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:55:56,664 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38383] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 02:55:56,664 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38383] zookeeper.ReadOnlyZKClient(139): Connect 0x6e22bfec to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:55:56,668 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@60e54c58, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:55:56,670 DEBUG [hconnection-0x64dad49c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:55:56,671 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40958, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-07-20 02:55:56,678 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38383] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 02:55:56,678 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,39331,1689821732285] 2023-07-20 02:55:56,680 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C39331%2C1689821732285.1689821733822] 2023-07-20 02:55:56,681 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,39331,1689821732285 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 02:55:56,681 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C39331%2C1689821732285 of peer 1-jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:56,782 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:56,785 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689821751552-0x10180a7ca410023, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689821751558.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,39331,1689821732285 2023-07-20 02:55:56,785 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285.replicationSource.shipperjenkins-hbase4.apache.org%2C39331%2C1689821732285,1-jenkins-hbase4.apache.org,39331,1689821732285] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,39331,1689821732285 with the following stats: Total replicated edits: 1, current progress: 2023-07-20 02:55:59,404 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,46185,1689821732333/1's WALs to 1689821751558.syncuptool.replication.org,1234,1 2023-07-20 02:55:59,406 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822 with data PBUF\x08\xFD\xAD\x01 2023-07-20 02:55:59,408 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,46185,1689821732333/1's WALs to 1689821751558.syncuptool.replication.org,1234,1 2023-07-20 02:56:00,559 INFO [jenkins-hbase4:38383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:56:00,559 INFO [jenkins-hbase4:38383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 1 2023-07-20 02:56:01,410 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46185,1689821732333] 2023-07-20 02:56:01,410 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46185,1689821732333] 2023-07-20 02:56:01,425 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46185,1689821732333] 2023-07-20 02:56:01,425 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,46185,1689821732333, ReplicationSource: 1, currentBandwidth=0 2023-07-20 02:56:01,425 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:56:01,450 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333] zookeeper.ReadOnlyZKClient(139): Connect 0x3a61c25d to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:01,455 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@27c4a8c8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:01,456 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:56:01,458 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:56:01,459 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10180a7ca41002a connected 2023-07-20 02:56:01,459 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,46185,1689821732333 (queues=1) is replicating from cluster=bbd9fa59-2213-4359-9606-c616f1cd40eb to cluster=69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:56:01,459 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C46185%2C1689821732333 2023-07-20 02:56:01,460 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,46185,1689821732333, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 02:56:01,461 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C46185%2C1689821732333 2023-07-20 02:56:01,469 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822 2023-07-20 02:56:01,469 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-20 02:56:01,469 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-20 02:56:01,470 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,46185,1689821732333 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs/jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822, lastWalPosition=30605, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-07-20 02:56:01,473 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:56:01,475 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36686, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:56:01,476 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38383] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 02:56:01,485 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38383] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 02:56:01,486 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46185,1689821732333] 2023-07-20 02:56:01,487 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C46185%2C1689821732333.1689821733822] 2023-07-20 02:56:01,488 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,46185,1689821732333 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 02:56:01,489 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C46185%2C1689821732333 of peer 1-jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:56:01,589 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:56:01,599 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689821751552-0x10180a7ca410023, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689821751558.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46185,1689821732333 2023-07-20 02:56:01,599 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333.replicationSource.shipperjenkins-hbase4.apache.org%2C46185%2C1689821732333,1-jenkins-hbase4.apache.org,46185,1689821732333] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,46185,1689821732333 with the following stats: Total replicated edits: 1, current progress: 2023-07-20 02:56:02,285 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 02:56:05,559 INFO [jenkins-hbase4:38383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:56:05,559 INFO [jenkins-hbase4:38383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-20 02:56:10,559 INFO [jenkins-hbase4:38383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:56:10,559 INFO [jenkins-hbase4:38383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-20 02:56:11,589 INFO [Listener at localhost/46633] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-20 02:56:11,692 DEBUG [Listener at localhost/46633.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410025, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:11,692 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x19e03ec9 to 127.0.0.1:55129 2023-07-20 02:56:11,693 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:11,898 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689821751552-0x10180a7ca410023, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:11,898 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689821751552-0x10180a7ca410023, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:11,911 INFO [Listener at localhost/46633] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-07-20 02:56:11,911 DEBUG [Listener at localhost/46633] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-07-20 02:56:11,913 INFO [Listener at localhost/46633] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-20 02:56:11,913 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:11,913 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:11,913 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:56:11,913 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:11,914 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:56:11,914 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:56:11,915 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36027 2023-07-20 02:56:11,915 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:56:11,916 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:56:11,917 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=master:36027 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:56:11,919 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:360270x0, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:56:11,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:36027-0x10180a7ca41002b connected 2023-07-20 02:56:11,923 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 02:56:11,923 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 02:56:11,923 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-20 02:56:11,927 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36027 2023-07-20 02:56:11,927 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36027 2023-07-20 02:56:11,927 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36027 2023-07-20 02:56:11,931 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36027 2023-07-20 02:56:11,931 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36027 2023-07-20 02:56:11,931 INFO [Listener at localhost/46633] master.HMaster(444): hbase.rootdir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61, hbase.cluster.distributed=false 2023-07-20 02:56:11,932 DEBUG [pool-543-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-07-20 02:56:11,943 INFO [Listener at localhost/46633] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-20 02:56:11,943 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:11,943 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:11,943 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:56:11,943 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:11,943 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:56:11,944 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:56:11,944 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41245 2023-07-20 02:56:11,945 INFO [Listener at localhost/46633] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 02:56:11,946 DEBUG [Listener at localhost/46633] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 02:56:11,947 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:56:11,948 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:56:11,949 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41245 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:56:11,952 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:412450x0, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:56:11,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41245-0x10180a7ca41002c connected 2023-07-20 02:56:11,955 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 02:56:11,955 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 02:56:11,956 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-20 02:56:11,957 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41245 2023-07-20 02:56:11,958 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41245 2023-07-20 02:56:11,960 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41245 2023-07-20 02:56:11,960 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41245 2023-07-20 02:56:11,962 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41245 2023-07-20 02:56:11,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:11,966 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-20 02:56:11,967 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:11,968 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-20 02:56:11,968 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-20 02:56:11,968 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:56:11,969 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-20 02:56:11,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,36027,1689821771912 from backup master directory 2023-07-20 02:56:11,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-20 02:56:11,972 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:11,972 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:56:11,972 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-20 02:56:11,972 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:11,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:56:12,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x37089bb6 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:12,027 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6061526c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:12,027 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 02:56:12,028 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-20 02:56:12,028 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:56:12,036 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36235,1689821731026 to hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36235,1689821731026-dead as it is dead 2023-07-20 02:56:12,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36235,1689821731026-dead/jenkins-hbase4.apache.org%2C36235%2C1689821731026.1689821732808 2023-07-20 02:56:12,038 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36235,1689821731026-dead/jenkins-hbase4.apache.org%2C36235%2C1689821731026.1689821732808 after 1ms 2023-07-20 02:56:12,039 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36235,1689821731026-dead/jenkins-hbase4.apache.org%2C36235%2C1689821731026.1689821732808 to hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36235%2C1689821731026.1689821732808 2023-07-20 02:56:12,039 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36235,1689821731026-dead 2023-07-20 02:56:12,039 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:12,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36027%2C1689821771912, suffix=, logDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36027,1689821771912, archiveDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/oldWALs, maxLogs=10 2023-07-20 02:56:12,059 DEBUG [RS-EventLoopGroup-15-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK] 2023-07-20 02:56:12,059 DEBUG [RS-EventLoopGroup-15-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK] 2023-07-20 02:56:12,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/WALs/jenkins-hbase4.apache.org,36027,1689821771912/jenkins-hbase4.apache.org%2C36027%2C1689821771912.1689821772042 2023-07-20 02:56:12,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK], DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK]] 2023-07-20 02:56:12,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:56:12,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:56:12,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:56:12,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:56:12,067 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:56:12,068 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-20 02:56:12,068 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-20 02:56:12,074 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3bab9e7494f1457c839490c938450f35 2023-07-20 02:56:12,074 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:12,075 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-20 02:56:12,075 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36235%2C1689821731026.1689821732808 2023-07-20 02:56:12,080 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36235%2C1689821731026.1689821732808 2023-07-20 02:56:12,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36235%2C1689821731026.1689821732808 2023-07-20 02:56:12,084 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:56:12,087 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-07-20 02:56:12,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74419282, jitterRate=0.1089337170124054}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 02:56:12,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:56:12,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-20 02:56:12,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-20 02:56:12,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-20 02:56:12,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-20 02:56:12,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-20 02:56:12,094 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-20 02:56:12,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-20 02:56:12,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-20 02:56:12,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-20 02:56:12,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-20 02:56:12,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-20 02:56:12,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-20 02:56:12,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-20 02:56:12,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-20 02:56:12,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-20 02:56:12,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46185,1689821732333, table=hbase:meta, region=1588230740 2023-07-20 02:56:12,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-07-20 02:56:12,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,39331,1689821732285 already deleted, retry=false 2023-07-20 02:56:12,100 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,39331,1689821732285 on jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:12,101 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,39331,1689821732285, splitWal=true, meta=false 2023-07-20 02:56:12,101 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,39331,1689821732285 (carryingMeta=false) jenkins-hbase4.apache.org,39331,1689821732285/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@181092a0[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 02:56:12,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,46185,1689821732333 already deleted, retry=false 2023-07-20 02:56:12,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46185,1689821732333 on jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:12,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46185,1689821732333, splitWal=true, meta=true 2023-07-20 02:56:12,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase4.apache.org,46185,1689821732333 (carryingMeta=true) jenkins-hbase4.apache.org,46185,1689821732333/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@6bc66dc6[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 02:56:12,114 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-20 02:56:12,114 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-20 02:56:12,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-20 02:56:12,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-20 02:56:12,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-20 02:56:12,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-20 02:56:12,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-20 02:56:12,120 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-20 02:56:12,121 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-20 02:56:12,121 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-20 02:56:12,121 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:56:12,122 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,36027,1689821771912, sessionid=0x10180a7ca41002b, setting cluster-up flag (Was=false) 2023-07-20 02:56:12,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-20 02:56:12,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:12,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-20 02:56:12,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:12,131 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/.hbase-snapshot/.tmp 2023-07-20 02:56:12,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:56:12,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:56:12,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:56:12,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:56:12,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-20 02:56:12,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:56:12,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,134 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689821802134 2023-07-20 02:56:12,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-20 02:56:12,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-20 02:56:12,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-20 02:56:12,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-20 02:56:12,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-20 02:56:12,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-20 02:56:12,136 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:12,136 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46185,1689821732333; numProcessing=1 2023-07-20 02:56:12,136 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39331,1689821732285; numProcessing=2 2023-07-20 02:56:12,136 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46185,1689821732333, splitWal=true, meta=true 2023-07-20 02:56:12,137 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,39331,1689821732285, splitWal=true, meta=false 2023-07-20 02:56:12,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-20 02:56:12,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-20 02:56:12,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-20 02:56:12,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-20 02:56:12,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-20 02:56:12,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821772138,5,FailOnTimeoutGroup] 2023-07-20 02:56:12,138 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46185,1689821732333, splitWal=true, meta=true, isMeta: true 2023-07-20 02:56:12,139 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821772138,5,FailOnTimeoutGroup] 2023-07-20 02:56:12,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:12,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-20 02:56:12,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:12,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:12,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689821772139, completionTime=-1 2023-07-20 02:56:12,139 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-20 02:56:12,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 02:56:12,140 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333-splitting 2023-07-20 02:56:12,141 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333-splitting dir is empty, no logs to split. 2023-07-20 02:56:12,141 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46185,1689821732333 WAL count=0, meta=true 2023-07-20 02:56:12,143 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333-splitting dir is empty, no logs to split. 2023-07-20 02:56:12,143 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46185,1689821732333 WAL count=0, meta=true 2023-07-20 02:56:12,143 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46185,1689821732333 WAL splitting is done? wals=0, meta=true 2023-07-20 02:56:12,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-20 02:56:12,145 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-20 02:56:12,146 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:56:12,164 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(951): ClusterId : bbd9fa59-2213-4359-9606-c616f1cd40eb 2023-07-20 02:56:12,164 DEBUG [RS:0;jenkins-hbase4:41245] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 02:56:12,167 DEBUG [RS:0;jenkins-hbase4:41245] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 02:56:12,167 DEBUG [RS:0;jenkins-hbase4:41245] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 02:56:12,169 DEBUG [RS:0;jenkins-hbase4:41245] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 02:56:12,171 DEBUG [RS:0;jenkins-hbase4:41245] zookeeper.ReadOnlyZKClient(139): Connect 0x599837d3 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:12,175 DEBUG [RS:0;jenkins-hbase4:41245] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@575fc290, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:12,175 DEBUG [RS:0;jenkins-hbase4:41245] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@53605ca3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:56:12,183 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:41245 2023-07-20 02:56:12,184 INFO [RS:0;jenkins-hbase4:41245] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 02:56:12,184 INFO [RS:0;jenkins-hbase4:41245] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 02:56:12,184 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 02:56:12,184 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36027,1689821771912 with isa=jenkins-hbase4.apache.org/172.31.14.131:41245, startcode=1689821771943 2023-07-20 02:56:12,184 DEBUG [RS:0;jenkins-hbase4:41245] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 02:56:12,186 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55775, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 02:56:12,188 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36027] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:12,188 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61 2023-07-20 02:56:12,188 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37763 2023-07-20 02:56:12,188 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 02:56:12,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 02:56:12,190 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 02:56:12,191 DEBUG [RS:0;jenkins-hbase4:41245] zookeeper.ZKUtil(162): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:12,191 WARN [RS:0;jenkins-hbase4:41245] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:56:12,191 INFO [RS:0;jenkins-hbase4:41245] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:56:12,191 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41245,1689821771943] 2023-07-20 02:56:12,192 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:12,207 INFO [RS:0;jenkins-hbase4:41245] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 02:56:12,208 DEBUG [RS:0;jenkins-hbase4:41245] zookeeper.ZKUtil(162): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:12,209 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:56:12,210 INFO [RS:0;jenkins-hbase4:41245] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 02:56:12,211 INFO [RS:0;jenkins-hbase4:41245] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 02:56:12,211 INFO [RS:0;jenkins-hbase4:41245] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 02:56:12,211 INFO [RS:0;jenkins-hbase4:41245] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:12,211 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 02:56:12,213 INFO [RS:0;jenkins-hbase4:41245] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:12,214 DEBUG [RS:0;jenkins-hbase4:41245] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,214 DEBUG [RS:0;jenkins-hbase4:41245] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,214 DEBUG [RS:0;jenkins-hbase4:41245] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,214 DEBUG [RS:0;jenkins-hbase4:41245] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,214 DEBUG [RS:0;jenkins-hbase4:41245] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,214 DEBUG [RS:0;jenkins-hbase4:41245] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:56:12,214 DEBUG [RS:0;jenkins-hbase4:41245] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,214 DEBUG [RS:0;jenkins-hbase4:41245] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,214 DEBUG [RS:0;jenkins-hbase4:41245] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,214 DEBUG [RS:0;jenkins-hbase4:41245] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:12,215 INFO [RS:0;jenkins-hbase4:41245] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:12,215 INFO [RS:0;jenkins-hbase4:41245] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:12,215 INFO [RS:0;jenkins-hbase4:41245] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:12,226 INFO [RS:0;jenkins-hbase4:41245] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 02:56:12,227 INFO [RS:0;jenkins-hbase4:41245] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41245,1689821771943-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:12,237 INFO [RS:0;jenkins-hbase4:41245] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-20 02:56:12,239 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1689821751558.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase4.apache.org,41245,1689821771943] 2023-07-20 02:56:12,250 INFO [RS:0;jenkins-hbase4:41245] regionserver.Replication(203): jenkins-hbase4.apache.org,41245,1689821771943 started 2023-07-20 02:56:12,250 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41245,1689821771943, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41245, sessionid=0x10180a7ca41002c 2023-07-20 02:56:12,250 DEBUG [RS:0;jenkins-hbase4:41245] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 02:56:12,250 DEBUG [RS:0;jenkins-hbase4:41245] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:12,250 DEBUG [RS:0;jenkins-hbase4:41245] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41245,1689821771943' 2023-07-20 02:56:12,250 DEBUG [RS:0;jenkins-hbase4:41245] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-20 02:56:12,250 DEBUG [RS:0;jenkins-hbase4:41245] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-20 02:56:12,251 DEBUG [RS:0;jenkins-hbase4:41245] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 02:56:12,251 DEBUG [RS:0;jenkins-hbase4:41245] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 02:56:12,251 DEBUG [RS:0;jenkins-hbase4:41245] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:12,251 DEBUG [RS:0;jenkins-hbase4:41245] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41245,1689821771943' 2023-07-20 02:56:12,251 DEBUG [RS:0;jenkins-hbase4:41245] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-20 02:56:12,251 DEBUG [RS:0;jenkins-hbase4:41245] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-20 02:56:12,251 DEBUG [RS:0;jenkins-hbase4:41245] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 02:56:12,251 INFO [RS:0;jenkins-hbase4:41245] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 02:56:12,251 INFO [RS:0;jenkins-hbase4:41245] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 02:56:12,260 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x414b7fab to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:12,265 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@51f51b6e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:12,266 INFO [RS:0;jenkins-hbase4:41245.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:56:12,268 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:56:12,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10180a7ca410030 connected 2023-07-20 02:56:12,271 INFO [RS:0;jenkins-hbase4:41245.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=bbd9fa59-2213-4359-9606-c616f1cd40eb to cluster=69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:56:12,296 DEBUG [jenkins-hbase4:36027] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 02:56:12,298 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41245,1689821771943, state=OPENING 2023-07-20 02:56:12,299 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-20 02:56:12,299 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-20 02:56:12,299 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41245,1689821771943}] 2023-07-20 02:56:12,353 INFO [RS:0;jenkins-hbase4:41245] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41245%2C1689821771943, suffix=, logDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,41245,1689821771943, archiveDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs, maxLogs=10 2023-07-20 02:56:12,371 DEBUG [RS-EventLoopGroup-15-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK] 2023-07-20 02:56:12,371 DEBUG [RS-EventLoopGroup-15-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK] 2023-07-20 02:56:12,380 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C41245%2C1689821771943 for peer 1 2023-07-20 02:56:12,380 INFO [RS:0;jenkins-hbase4:41245] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,41245,1689821771943/jenkins-hbase4.apache.org%2C41245%2C1689821771943.1689821772354 2023-07-20 02:56:12,380 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C41245%2C1689821771943 2023-07-20 02:56:12,380 INFO [RS:0;jenkins-hbase4:41245] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 02:56:12,381 DEBUG [RS:0;jenkins-hbase4:41245] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK], DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK]] 2023-07-20 02:56:12,381 INFO [RS:0;jenkins-hbase4:41245.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C41245%2C1689821771943 2023-07-20 02:56:12,385 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:12,457 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:12,457 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:56:12,459 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55974, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:56:12,463 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-20 02:56:12,463 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:56:12,465 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41245%2C1689821771943.meta, suffix=.meta, logDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,41245,1689821771943, archiveDir=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs, maxLogs=10 2023-07-20 02:56:12,480 DEBUG [RS-EventLoopGroup-15-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK] 2023-07-20 02:56:12,480 DEBUG [RS-EventLoopGroup-15-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK] 2023-07-20 02:56:12,482 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,41245,1689821771943/jenkins-hbase4.apache.org%2C41245%2C1689821771943.meta.1689821772465.meta 2023-07-20 02:56:12,482 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44997,DS-3b140a78-8011-4c98-96ec-6684fce7ecc8,DISK], DatanodeInfoWithStorage[127.0.0.1:40993,DS-bb6f101f-116f-465d-8349-dd6666cb59ca,DISK]] 2023-07-20 02:56:12,482 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:56:12,483 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-20 02:56:12,483 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-20 02:56:12,483 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-20 02:56:12,483 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-20 02:56:12,483 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:56:12,483 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-20 02:56:12,483 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-20 02:56:12,485 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 02:56:12,486 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/info 2023-07-20 02:56:12,486 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/info 2023-07-20 02:56:12,486 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 02:56:12,493 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:12,496 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/info/da07c5a494c2484a8c01dd3b447a87ae 2023-07-20 02:56:12,497 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:12,497 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 02:56:12,498 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:56:12,498 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:56:12,498 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 02:56:12,504 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/rep_barrier/7c143364f9ad48829a6ce0d77b4e3124 2023-07-20 02:56:12,505 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:12,505 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 02:56:12,506 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/table 2023-07-20 02:56:12,506 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/table 2023-07-20 02:56:12,506 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 02:56:12,513 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/table/03dd25f3e2864c21837ddbdf1e79a2f0 2023-07-20 02:56:12,514 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:12,515 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740 2023-07-20 02:56:12,516 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740 2023-07-20 02:56:12,518 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 02:56:12,519 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 02:56:12,521 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63411874, jitterRate=-0.05508944392204285}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 02:56:12,521 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 02:56:12,521 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1689821772457 2023-07-20 02:56:12,527 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-20 02:56:12,528 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-20 02:56:12,528 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41245,1689821771943, state=OPEN 2023-07-20 02:56:12,531 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-20 02:56:12,531 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-20 02:56:12,533 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-07-20 02:56:12,533 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41245,1689821771943 in 232 msec 2023-07-20 02:56:12,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-20 02:56:12,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 389 msec 2023-07-20 02:56:12,598 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:12,703 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:12,808 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:12,913 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:13,018 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:13,123 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:13,227 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:13,332 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:13,437 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:13,541 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:13,646 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:13,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1553ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-20 02:56:13,751 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:13,857 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:13,962 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:14,067 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:14,172 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:14,277 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:14,381 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:14,486 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:14,591 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:14,695 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:14,800 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:14,905 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:15,010 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:15,114 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:15,194 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-20 02:56:15,219 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:15,324 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:15,429 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:15,534 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:15,559 INFO [jenkins-hbase4:38383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:56:15,559 INFO [jenkins-hbase4:38383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-20 02:56:15,638 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:15,744 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:15,848 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:15,953 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:16,058 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:16,163 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:16,267 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:16,372 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:16,477 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:16,581 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:16,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-20 02:56:16,647 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-20 02:56:16,650 DEBUG [hconnection-0x674a64c7-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:56:16,651 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55976, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:56:16,656 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=1af5feb4e30a65bed26cf0457e90777d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46185,1689821732333, regionLocation=jenkins-hbase4.apache.org,46185,1689821732333, openSeqNum=2 2023-07-20 02:56:16,656 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=300903cec0fd231ca88313bcf649662a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39331,1689821732285, regionLocation=jenkins-hbase4.apache.org,39331,1689821732285, openSeqNum=2 2023-07-20 02:56:16,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=392fe9dfffd6a3888f97c341c2952aeb, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46185,1689821732333, regionLocation=jenkins-hbase4.apache.org,46185,1689821732333, openSeqNum=2 2023-07-20 02:56:16,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-20 02:56:16,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689821836657 2023-07-20 02:56:16,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689821896657 2023-07-20 02:56:16,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 10 msec 2023-07-20 02:56:16,673 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46185,1689821732333 had 3 regions 2023-07-20 02:56:16,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36027,1689821771912-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:16,673 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,39331,1689821732285 had 1 regions 2023-07-20 02:56:16,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36027,1689821771912-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:16,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36027,1689821771912-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:16,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:36027, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:16,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:16,673 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. is NOT online; state={1af5feb4e30a65bed26cf0457e90777d state=OPEN, ts=1689821776656, server=jenkins-hbase4.apache.org,46185,1689821732333}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-20 02:56:16,675 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46185,1689821732333, splitWal=true, meta=true, isMeta: false 2023-07-20 02:56:16,675 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,39331,1689821732285, splitWal=true, meta=false, isMeta: false 2023-07-20 02:56:16,678 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333-splitting dir is empty, no logs to split. 2023-07-20 02:56:16,678 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,46185,1689821732333 WAL count=0, meta=false 2023-07-20 02:56:16,678 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,46185,1689821732333/hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d., unknown_server=jenkins-hbase4.apache.org,39331,1689821732285/t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a., unknown_server=jenkins-hbase4.apache.org,46185,1689821732333/t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:56:16,678 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285-splitting 2023-07-20 02:56:16,679 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285-splitting dir is empty, no logs to split. 2023-07-20 02:56:16,679 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,39331,1689821732285 WAL count=0, meta=false 2023-07-20 02:56:16,680 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,46185,1689821732333-splitting dir is empty, no logs to split. 2023-07-20 02:56:16,680 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,46185,1689821732333 WAL count=0, meta=false 2023-07-20 02:56:16,680 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46185,1689821732333 WAL splitting is done? wals=0, meta=false 2023-07-20 02:56:16,681 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285-splitting dir is empty, no logs to split. 2023-07-20 02:56:16,681 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,39331,1689821732285 WAL count=0, meta=false 2023-07-20 02:56:16,681 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,39331,1689821732285 WAL splitting is done? wals=0, meta=false 2023-07-20 02:56:16,682 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=392fe9dfffd6a3888f97c341c2952aeb, ASSIGN}, {pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1af5feb4e30a65bed26cf0457e90777d, ASSIGN}] 2023-07-20 02:56:16,684 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,39331,1689821732285 failed, ignore...File hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,39331,1689821732285-splitting does not exist. 2023-07-20 02:56:16,685 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=300903cec0fd231ca88313bcf649662a, ASSIGN}] 2023-07-20 02:56:16,685 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1af5feb4e30a65bed26cf0457e90777d, ASSIGN 2023-07-20 02:56:16,685 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=392fe9dfffd6a3888f97c341c2952aeb, ASSIGN 2023-07-20 02:56:16,686 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=1af5feb4e30a65bed26cf0457e90777d, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:56:16,686 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=300903cec0fd231ca88313bcf649662a, ASSIGN 2023-07-20 02:56:16,686 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=392fe9dfffd6a3888f97c341c2952aeb, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:56:16,687 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:16,687 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=300903cec0fd231ca88313bcf649662a, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:56:16,791 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:16,836 DEBUG [jenkins-hbase4:36027] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 02:56:16,839 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=392fe9dfffd6a3888f97c341c2952aeb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:16,839 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=1af5feb4e30a65bed26cf0457e90777d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:16,839 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821776839"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821776839"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821776839"}]},"ts":"1689821776839"} 2023-07-20 02:56:16,839 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=300903cec0fd231ca88313bcf649662a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:16,839 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821776839"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821776839"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821776839"}]},"ts":"1689821776839"} 2023-07-20 02:56:16,839 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821776839"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821776839"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821776839"}]},"ts":"1689821776839"} 2023-07-20 02:56:16,842 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure 392fe9dfffd6a3888f97c341c2952aeb, server=jenkins-hbase4.apache.org,41245,1689821771943}] 2023-07-20 02:56:16,843 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure 1af5feb4e30a65bed26cf0457e90777d, server=jenkins-hbase4.apache.org,41245,1689821771943}] 2023-07-20 02:56:16,844 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=24, state=RUNNABLE; OpenRegionProcedure 300903cec0fd231ca88313bcf649662a, server=jenkins-hbase4.apache.org,41245,1689821771943}] 2023-07-20 02:56:16,896 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:16,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:56:16,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 300903cec0fd231ca88313bcf649662a, NAME => 't1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:56:16,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 300903cec0fd231ca88313bcf649662a 2023-07-20 02:56:16,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:56:16,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 300903cec0fd231ca88313bcf649662a 2023-07-20 02:56:16,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 300903cec0fd231ca88313bcf649662a 2023-07-20 02:56:17,000 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 02:56:17,001 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 300903cec0fd231ca88313bcf649662a 2023-07-20 02:56:17,001 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:56:17,001 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1af5feb4e30a65bed26cf0457e90777d, NAME => 'hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:56:17,002 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:56:17,002 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:56:17,002 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:56:17,002 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:56:17,002 DEBUG [StoreOpener-300903cec0fd231ca88313bcf649662a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/cf1 2023-07-20 02:56:17,002 DEBUG [StoreOpener-300903cec0fd231ca88313bcf649662a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/cf1 2023-07-20 02:56:17,002 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 300903cec0fd231ca88313bcf649662a columnFamilyName cf1 2023-07-20 02:56:17,003 INFO [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:56:17,004 DEBUG [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/info 2023-07-20 02:56:17,004 DEBUG [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/info 2023-07-20 02:56:17,005 INFO [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1af5feb4e30a65bed26cf0457e90777d columnFamilyName info 2023-07-20 02:56:17,009 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b972b96393274eb18c7163b9cb64372d 2023-07-20 02:56:17,009 DEBUG [StoreOpener-300903cec0fd231ca88313bcf649662a-1] regionserver.HStore(539): loaded hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/cf1/b972b96393274eb18c7163b9cb64372d 2023-07-20 02:56:17,010 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] regionserver.HStore(310): Store=300903cec0fd231ca88313bcf649662a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:17,010 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 300903cec0fd231ca88313bcf649662a 2023-07-20 02:56:17,011 DEBUG [StoreOpener-300903cec0fd231ca88313bcf649662a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/norep 2023-07-20 02:56:17,011 DEBUG [StoreOpener-300903cec0fd231ca88313bcf649662a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/norep 2023-07-20 02:56:17,011 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 300903cec0fd231ca88313bcf649662a columnFamilyName norep 2023-07-20 02:56:17,011 DEBUG [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] regionserver.HStore(539): loaded hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/info/86eea25cb55d41a68aa2a9f148b9ec9d 2023-07-20 02:56:17,011 INFO [StoreOpener-1af5feb4e30a65bed26cf0457e90777d-1] regionserver.HStore(310): Store=1af5feb4e30a65bed26cf0457e90777d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:17,012 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:56:17,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:56:17,017 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2efb46b211354194bf6677b74b8dba05 2023-07-20 02:56:17,017 DEBUG [StoreOpener-300903cec0fd231ca88313bcf649662a-1] regionserver.HStore(539): loaded hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/norep/2efb46b211354194bf6677b74b8dba05 2023-07-20 02:56:17,017 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:56:17,017 INFO [StoreOpener-300903cec0fd231ca88313bcf649662a-1] regionserver.HStore(310): Store=300903cec0fd231ca88313bcf649662a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:17,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a 2023-07-20 02:56:17,018 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1af5feb4e30a65bed26cf0457e90777d; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63161235, jitterRate=-0.05882425606250763}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 02:56:17,018 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1af5feb4e30a65bed26cf0457e90777d: 2023-07-20 02:56:17,019 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d., pid=26, masterSystemTime=1689821776994 2023-07-20 02:56:17,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a 2023-07-20 02:56:17,022 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:56:17,022 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:56:17,022 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=1af5feb4e30a65bed26cf0457e90777d, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:17,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 02:56:17,023 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821777022"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821777022"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821777022"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821777022"}]},"ts":"1689821777022"} 2023-07-20 02:56:17,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 300903cec0fd231ca88313bcf649662a 2023-07-20 02:56:17,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 300903cec0fd231ca88313bcf649662a; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69970286, jitterRate=0.04263851046562195}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 02:56:17,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 300903cec0fd231ca88313bcf649662a: 2023-07-20 02:56:17,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a., pid=27, masterSystemTime=1689821776994 2023-07-20 02:56:17,026 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-07-20 02:56:17,026 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure 1af5feb4e30a65bed26cf0457e90777d, server=jenkins-hbase4.apache.org,41245,1689821771943 in 181 msec 2023-07-20 02:56:17,027 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=1af5feb4e30a65bed26cf0457e90777d, ASSIGN in 344 msec 2023-07-20 02:56:17,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:56:17,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:56:17,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:56:17,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 392fe9dfffd6a3888f97c341c2952aeb, NAME => 't2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:56:17,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:56:17,028 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=300903cec0fd231ca88313bcf649662a, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:17,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:56:17,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:56:17,028 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821777028"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821777028"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821777028"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821777028"}]},"ts":"1689821777028"} 2023-07-20 02:56:17,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:56:17,030 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:56:17,031 DEBUG [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/cf1 2023-07-20 02:56:17,031 DEBUG [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/cf1 2023-07-20 02:56:17,031 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=24 2023-07-20 02:56:17,031 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 392fe9dfffd6a3888f97c341c2952aeb columnFamilyName cf1 2023-07-20 02:56:17,031 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=24, state=SUCCESS; OpenRegionProcedure 300903cec0fd231ca88313bcf649662a, server=jenkins-hbase4.apache.org,41245,1689821771943 in 186 msec 2023-07-20 02:56:17,033 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=18 2023-07-20 02:56:17,033 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,39331,1689821732285 after splitting done 2023-07-20 02:56:17,033 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=300903cec0fd231ca88313bcf649662a, ASSIGN in 346 msec 2023-07-20 02:56:17,033 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,39331,1689821732285 from processing; numProcessing=1 2023-07-20 02:56:17,034 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,39331,1689821732285, splitWal=true, meta=false in 4.9330 sec 2023-07-20 02:56:17,038 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 29c24efb07ad45499008086172519b02 2023-07-20 02:56:17,038 DEBUG [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] regionserver.HStore(539): loaded hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/cf1/29c24efb07ad45499008086172519b02 2023-07-20 02:56:17,038 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] regionserver.HStore(310): Store=392fe9dfffd6a3888f97c341c2952aeb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:17,038 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:56:17,039 DEBUG [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/norep 2023-07-20 02:56:17,039 DEBUG [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/norep 2023-07-20 02:56:17,040 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 392fe9dfffd6a3888f97c341c2952aeb columnFamilyName norep 2023-07-20 02:56:17,046 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4a88f5547efe4f2eb883aa25fb8a0014 2023-07-20 02:56:17,046 DEBUG [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] regionserver.HStore(539): loaded hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/norep/4a88f5547efe4f2eb883aa25fb8a0014 2023-07-20 02:56:17,046 INFO [StoreOpener-392fe9dfffd6a3888f97c341c2952aeb-1] regionserver.HStore(310): Store=392fe9dfffd6a3888f97c341c2952aeb/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:17,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:56:17,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:56:17,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 02:56:17,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:56:17,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 392fe9dfffd6a3888f97c341c2952aeb; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61500882, jitterRate=-0.08356544375419617}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 02:56:17,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 392fe9dfffd6a3888f97c341c2952aeb: 2023-07-20 02:56:17,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb., pid=25, masterSystemTime=1689821776994 2023-07-20 02:56:17,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:56:17,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:56:17,055 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=392fe9dfffd6a3888f97c341c2952aeb, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:17,055 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821777055"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821777055"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821777055"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821777055"}]},"ts":"1689821777055"} 2023-07-20 02:56:17,058 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-20 02:56:17,058 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure 392fe9dfffd6a3888f97c341c2952aeb, server=jenkins-hbase4.apache.org,41245,1689821771943 in 214 msec 2023-07-20 02:56:17,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=19 2023-07-20 02:56:17,059 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46185,1689821732333 after splitting done 2023-07-20 02:56:17,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=392fe9dfffd6a3888f97c341c2952aeb, ASSIGN in 376 msec 2023-07-20 02:56:17,060 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46185,1689821732333 from processing; numProcessing=0 2023-07-20 02:56:17,061 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46185,1689821732333, splitWal=true, meta=true in 4.9570 sec 2023-07-20 02:56:17,105 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:17,105 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 02:56:17,105 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:17,105 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,41245,1689821771943/jenkins-hbase4.apache.org%2C41245%2C1689821771943.1689821772354, lastWalPosition=1162, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 02:56:17,210 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:17,250 INFO [jenkins-hbase4:41245Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C41245%2C1689821771943]: currently replicating from: hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,41245,1689821771943/jenkins-hbase4.apache.org%2C41245%2C1689821771943.1689821772354 at position: 1162 2023-07-20 02:56:17,269 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 02:56:17,317 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:17,422 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:17,527 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:17,632 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:17,674 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-07-20 02:56:17,694 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-20 02:56:17,696 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-20 02:56:17,696 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.724sec 2023-07-20 02:56:17,697 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-20 02:56:17,697 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-20 02:56:17,697 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-20 02:56:17,697 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36027,1689821771912-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-20 02:56:17,697 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36027,1689821771912-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-20 02:56:17,700 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-20 02:56:17,736 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:17,768 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(139): Connect 0x644ebd97 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:17,773 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3228d4bc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:17,775 DEBUG [hconnection-0x759ee5d7-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:56:17,777 INFO [RS-EventLoopGroup-15-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55990, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:56:17,782 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-20 02:56:17,783 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x644ebd97 to 127.0.0.1:55129 2023-07-20 02:56:17,783 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:17,784 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:36027 after: jenkins-hbase4.apache.org:36027 2023-07-20 02:56:17,784 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(139): Connect 0x41605d54 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:17,788 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4352b8f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:17,789 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5dcc4257 to 127.0.0.1:55129 2023-07-20 02:56:17,789 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:17,789 DEBUG [Listener at localhost/46633] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-20 02:56:17,789 DEBUG [Listener at localhost/46633] util.JVMClusterUtil(257): Found active master hash=1278911200, stopped=false 2023-07-20 02:56:17,789 INFO [Listener at localhost/46633] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:56:17,791 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 02:56:17,791 INFO [Listener at localhost/46633] procedure2.ProcedureExecutor(629): Stopping 2023-07-20 02:56:17,791 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:56:17,791 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38383-0x10180a7ca41001e, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 02:56:17,792 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x520f6998 to 127.0.0.1:55129 2023-07-20 02:56:17,793 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:17,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:56:17,794 INFO [Listener at localhost/46633] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38383,1689821745167' ***** 2023-07-20 02:56:17,794 INFO [Listener at localhost/46633] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 02:56:17,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38383-0x10180a7ca41001e, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:56:17,794 INFO [RS:0;jenkins-hbase4:38383] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 02:56:17,794 INFO [RS:0;jenkins-hbase4:38383] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 02:56:17,794 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 02:56:17,794 INFO [RS:0;jenkins-hbase4:38383] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 02:56:17,795 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(3305): Received CLOSE for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:17,795 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(3305): Received CLOSE for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:56:17,795 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(3305): Received CLOSE for 6908f43230333d97246933b2af61e211 2023-07-20 02:56:17,795 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:56:17,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 956fe6fd22cc055b7d5c09ea7e2fd4a7, disabling compactions & flushes 2023-07-20 02:56:17,796 DEBUG [RS:0;jenkins-hbase4:38383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x600bf1f0 to 127.0.0.1:55129 2023-07-20 02:56:17,796 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:17,796 DEBUG [RS:0;jenkins-hbase4:38383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:17,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:17,796 INFO [RS:0;jenkins-hbase4:38383] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 02:56:17,796 INFO [RS:0;jenkins-hbase4:38383] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 02:56:17,796 INFO [RS:0;jenkins-hbase4:38383] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 02:56:17,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. after waiting 0 ms 2023-07-20 02:56:17,796 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-20 02:56:17,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:17,797 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-20 02:56:17,797 DEBUG [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1478): Online Regions={956fe6fd22cc055b7d5c09ea7e2fd4a7=hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7., 80b6625deaec89ca31a4c1b65e4a6544=t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544., 1588230740=hbase:meta,,1.1588230740, 6908f43230333d97246933b2af61e211=t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.} 2023-07-20 02:56:17,797 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 02:56:17,797 DEBUG [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1504): Waiting on 1588230740, 6908f43230333d97246933b2af61e211, 80b6625deaec89ca31a4c1b65e4a6544, 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:17,797 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 02:56:17,797 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 02:56:17,797 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 02:56:17,797 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 02:56:17,799 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-20 02:56:17,806 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-20 02:56:17,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:17,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 956fe6fd22cc055b7d5c09ea7e2fd4a7: 2023-07-20 02:56:17,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:17,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 80b6625deaec89ca31a4c1b65e4a6544, disabling compactions & flushes 2023-07-20 02:56:17,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:17,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:17,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. after waiting 0 ms 2023-07-20 02:56:17,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:17,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 80b6625deaec89ca31a4c1b65e4a6544 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-07-20 02:56:17,823 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/.tmp/info/96735ff5a6a9432c98af6ae4fc0b5b78 2023-07-20 02:56:17,834 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/.tmp/info/96735ff5a6a9432c98af6ae4fc0b5b78 as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info/96735ff5a6a9432c98af6ae4fc0b5b78 2023-07-20 02:56:17,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/.tmp/cf1/12d666253300492d8b93c8e4058ffad7 2023-07-20 02:56:17,842 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:17,842 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 12d666253300492d8b93c8e4058ffad7 2023-07-20 02:56:17,843 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info/96735ff5a6a9432c98af6ae4fc0b5b78, entries=24, sequenceid=31, filesize=7.4 K 2023-07-20 02:56:17,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/.tmp/cf1/12d666253300492d8b93c8e4058ffad7 as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1/12d666253300492d8b93c8e4058ffad7 2023-07-20 02:56:17,844 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 46ms, sequenceid=31, compaction requested=false 2023-07-20 02:56:17,856 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-20 02:56:17,857 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 02:56:17,858 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 02:56:17,858 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 02:56:17,858 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-20 02:56:17,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 12d666253300492d8b93c8e4058ffad7 2023-07-20 02:56:17,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1/12d666253300492d8b93c8e4058ffad7, entries=50, sequenceid=14, filesize=6.4 K 2023-07-20 02:56:17,859 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for 80b6625deaec89ca31a4c1b65e4a6544 in 51ms, sequenceid=14, compaction requested=false 2023-07-20 02:56:17,863 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:56:17,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-07-20 02:56:17,872 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:17,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 80b6625deaec89ca31a4c1b65e4a6544: 2023-07-20 02:56:17,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:17,873 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6908f43230333d97246933b2af61e211, disabling compactions & flushes 2023-07-20 02:56:17,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:17,873 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:17,873 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. after waiting 0 ms 2023-07-20 02:56:17,873 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:17,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6908f43230333d97246933b2af61e211 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-07-20 02:56:17,898 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=15 (bloomFilter=true), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/.tmp/cf1/ccb9665ed0cc48858e87b550c7de02a8 2023-07-20 02:56:17,905 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ccb9665ed0cc48858e87b550c7de02a8 2023-07-20 02:56:17,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/.tmp/cf1/ccb9665ed0cc48858e87b550c7de02a8 as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1/ccb9665ed0cc48858e87b550c7de02a8 2023-07-20 02:56:17,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ccb9665ed0cc48858e87b550c7de02a8 2023-07-20 02:56:17,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1/ccb9665ed0cc48858e87b550c7de02a8, entries=100, sequenceid=15, filesize=7.9 K 2023-07-20 02:56:17,914 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for 6908f43230333d97246933b2af61e211 in 41ms, sequenceid=15, compaction requested=false 2023-07-20 02:56:17,924 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/recovered.edits/18.seqid, newMaxSeqId=18, maxSeqId=11 2023-07-20 02:56:17,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:17,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6908f43230333d97246933b2af61e211: 2023-07-20 02:56:17,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:17,947 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:17,997 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38383,1689821745167; all regions closed. 2023-07-20 02:56:18,003 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167/jenkins-hbase4.apache.org%2C38383%2C1689821745167.meta.1689821745746.meta not finished, retry = 0 2023-07-20 02:56:18,052 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:18,106 DEBUG [RS:0;jenkins-hbase4:38383] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs 2023-07-20 02:56:18,107 INFO [RS:0;jenkins-hbase4:38383] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38383%2C1689821745167.meta:.meta(num 1689821745746) 2023-07-20 02:56:18,113 DEBUG [RS:0;jenkins-hbase4:38383] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs 2023-07-20 02:56:18,114 INFO [RS:0;jenkins-hbase4:38383] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38383%2C1689821745167:(num 1689821745665) 2023-07-20 02:56:18,114 DEBUG [RS:0;jenkins-hbase4:38383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:18,114 INFO [RS:0;jenkins-hbase4:38383] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:56:18,114 INFO [RS:0;jenkins-hbase4:38383] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 02:56:18,114 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:56:18,114 DEBUG [RS:0;jenkins-hbase4:38383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6e22bfec to 127.0.0.1:55129 2023-07-20 02:56:18,114 DEBUG [RS:0;jenkins-hbase4:38383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:18,115 INFO [RS:0;jenkins-hbase4:38383] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38383 2023-07-20 02:56:18,120 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:18,120 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38383-0x10180a7ca41001e, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,38383,1689821745167 2023-07-20 02:56:18,120 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410027, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:18,120 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca41002a, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:18,120 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38383-0x10180a7ca41001e, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:18,120 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:56:18,120 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:56:18,121 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38383,1689821745167] 2023-07-20 02:56:18,121 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38383,1689821745167; numProcessing=1 2023-07-20 02:56:18,122 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,38383,1689821745167 already deleted, retry=false 2023-07-20 02:56:18,122 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38383,1689821745167 expired; onlineServers=0 2023-07-20 02:56:18,122 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40859,1689821745131' ***** 2023-07-20 02:56:18,122 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-20 02:56:18,123 DEBUG [M:0;jenkins-hbase4:40859] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@48052fbf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:56:18,123 INFO [M:0;jenkins-hbase4:40859] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40859,1689821745131 2023-07-20 02:56:18,123 INFO [M:0;jenkins-hbase4:40859] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40859,1689821745131; all regions closed. 2023-07-20 02:56:18,123 DEBUG [M:0;jenkins-hbase4:40859] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:18,123 DEBUG [M:0;jenkins-hbase4:40859] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-20 02:56:18,123 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-20 02:56:18,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821745416] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821745416,5,FailOnTimeoutGroup] 2023-07-20 02:56:18,123 DEBUG [M:0;jenkins-hbase4:40859] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-20 02:56:18,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821745416] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821745416,5,FailOnTimeoutGroup] 2023-07-20 02:56:18,125 INFO [M:0;jenkins-hbase4:40859] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-20 02:56:18,126 INFO [M:0;jenkins-hbase4:40859] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-20 02:56:18,126 INFO [M:0;jenkins-hbase4:40859] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-20 02:56:18,126 DEBUG [M:0;jenkins-hbase4:40859] master.HMaster(1512): Stopping service threads 2023-07-20 02:56:18,126 INFO [M:0;jenkins-hbase4:40859] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-20 02:56:18,127 ERROR [M:0;jenkins-hbase4:40859] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-20 02:56:18,127 INFO [M:0;jenkins-hbase4:40859] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-20 02:56:18,127 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-20 02:56:18,127 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-20 02:56:18,127 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:56:18,127 DEBUG [M:0;jenkins-hbase4:40859] zookeeper.ZKUtil(398): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-20 02:56:18,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:56:18,127 WARN [M:0;jenkins-hbase4:40859] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-20 02:56:18,127 INFO [M:0;jenkins-hbase4:40859] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-20 02:56:18,128 INFO [M:0;jenkins-hbase4:40859] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-20 02:56:18,129 DEBUG [M:0;jenkins-hbase4:40859] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 02:56:18,129 INFO [M:0;jenkins-hbase4:40859] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:18,129 DEBUG [M:0;jenkins-hbase4:40859] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:18,129 DEBUG [M:0;jenkins-hbase4:40859] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 02:56:18,129 DEBUG [M:0;jenkins-hbase4:40859] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:18,129 INFO [M:0;jenkins-hbase4:40859] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=36.37 KB heapSize=46.74 KB 2023-07-20 02:56:18,146 INFO [M:0;jenkins-hbase4:40859] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=36.37 KB at sequenceid=238 (bloomFilter=true), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/adfd5ca96e9945a5b803a00e00421eac 2023-07-20 02:56:18,151 INFO [M:0;jenkins-hbase4:40859] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for adfd5ca96e9945a5b803a00e00421eac 2023-07-20 02:56:18,153 DEBUG [M:0;jenkins-hbase4:40859] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/adfd5ca96e9945a5b803a00e00421eac as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/adfd5ca96e9945a5b803a00e00421eac 2023-07-20 02:56:18,156 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:18,158 INFO [M:0;jenkins-hbase4:40859] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for adfd5ca96e9945a5b803a00e00421eac 2023-07-20 02:56:18,159 INFO [M:0;jenkins-hbase4:40859] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/adfd5ca96e9945a5b803a00e00421eac, entries=20, sequenceid=238, filesize=5.6 K 2023-07-20 02:56:18,160 INFO [M:0;jenkins-hbase4:40859] regionserver.HRegion(2948): Finished flush of dataSize ~36.37 KB/37244, heapSize ~46.73 KB/47848, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 31ms, sequenceid=238, compaction requested=false 2023-07-20 02:56:18,163 INFO [M:0;jenkins-hbase4:40859] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:18,163 DEBUG [M:0;jenkins-hbase4:40859] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:56:18,168 INFO [M:0;jenkins-hbase4:40859] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-20 02:56:18,168 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:56:18,169 INFO [M:0;jenkins-hbase4:40859] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40859 2023-07-20 02:56:18,173 DEBUG [M:0;jenkins-hbase4:40859] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,40859,1689821745131 already deleted, retry=false 2023-07-20 02:56:18,261 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:18,289 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:18,289 INFO [M:0;jenkins-hbase4:40859] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40859,1689821745131; zookeeper connection closed. 2023-07-20 02:56:18,289 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:40859-0x10180a7ca41001d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:18,366 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-20 02:56:18,389 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38383-0x10180a7ca41001e, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:18,389 INFO [RS:0;jenkins-hbase4:38383] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38383,1689821745167; zookeeper connection closed. 2023-07-20 02:56:18,389 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:38383-0x10180a7ca41001e, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:18,390 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@70d59832] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@70d59832 2023-07-20 02:56:18,391 INFO [Listener at localhost/46633] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-20 02:56:18,392 DEBUG [hconnection-0x35058061-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:56:18,394 INFO [RS-EventLoopGroup-15-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55998, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:56:18,475 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 10030 2023-07-20 02:56:18,475 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.ReplicationSourceWALReader(162): Read 84 WAL entries eligible for replication 2023-07-20 02:56:18,475 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 10030 2023-07-20 02:56:18,476 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/300903cec0fd231ca88313bcf649662a/112=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/113=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/114=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/115=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/116=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/117=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/118=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/119=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/120=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/121=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/122=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/123=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/124=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/125=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/126=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/127=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/128=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/129=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/130=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/131=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/132=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/133=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/134=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/135=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/136=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/137=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/138=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/139=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/140=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/141=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/142=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/143=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/144=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/145=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/146=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/147=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/148=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/149=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/150=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/151=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/152=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/153=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/154=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/155=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/156=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/157=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/158=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/159=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/160=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/161=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/162=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/163=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/164=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/165=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/166=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/167=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/168=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/169=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/170=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/171=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/172=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/173=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/174=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/175=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/176=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/177=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/178=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/179=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/180=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/181=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/182=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/183=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/184=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/185=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/186=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/187=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/188=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/189=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/190=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/191=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/192=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/193=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/194=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/195=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/WALs/jenkins-hbase4.apache.org,41245,1689821771943/jenkins-hbase4.apache.org%2C41245%2C1689821771943.1689821772354, lastWalPosition=10030, nbRowKeys=84, nbHFiles=0, heapSize=17472, lastSeqIds={}, endOfFile=false,usedBufferSize=17472] 2023-07-20 02:56:18,479 WARN [RS:0;jenkins-hbase4:41245.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-20 02:56:18,580 WARN [RS:0;jenkins-hbase4:41245.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-20 02:56:18,588 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 23174 2023-07-20 02:56:18,588 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.ReplicationSourceWALReader(162): Read 123 WAL entries eligible for replication 2023-07-20 02:56:18,588 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 23174 2023-07-20 02:56:18,666 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x41605d54 to 127.0.0.1:55129 2023-07-20 02:56:18,666 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:18,666 DEBUG [Listener at localhost/46633] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-20 02:56:18,667 DEBUG [Listener at localhost/46633] util.JVMClusterUtil(257): Found active master hash=1694701683, stopped=false 2023-07-20 02:56:18,667 INFO [Listener at localhost/46633] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:18,669 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-20 02:56:18,669 INFO [Listener at localhost/46633] procedure2.ProcedureExecutor(629): Stopping 2023-07-20 02:56:18,669 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-20 02:56:18,670 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:56:18,672 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x37089bb6 to 127.0.0.1:55129 2023-07-20 02:56:18,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 02:56:18,672 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:18,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 02:56:18,673 INFO [Listener at localhost/46633] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41245,1689821771943' ***** 2023-07-20 02:56:18,673 INFO [Listener at localhost/46633] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 02:56:18,673 INFO [RS:0;jenkins-hbase4:41245] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 02:56:18,673 INFO [RS:0;jenkins-hbase4:41245] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 02:56:18,673 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 02:56:18,673 INFO [RS:0;jenkins-hbase4:41245] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 02:56:18,674 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(3305): Received CLOSE for 1af5feb4e30a65bed26cf0457e90777d 2023-07-20 02:56:18,674 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(3305): Received CLOSE for 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:56:18,674 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(3305): Received CLOSE for 300903cec0fd231ca88313bcf649662a 2023-07-20 02:56:18,674 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:18,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1af5feb4e30a65bed26cf0457e90777d, disabling compactions & flushes 2023-07-20 02:56:18,675 DEBUG [RS:0;jenkins-hbase4:41245] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x599837d3 to 127.0.0.1:55129 2023-07-20 02:56:18,675 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:56:18,675 DEBUG [RS:0;jenkins-hbase4:41245] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:18,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:56:18,675 INFO [RS:0;jenkins-hbase4:41245] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 02:56:18,675 INFO [RS:0;jenkins-hbase4:41245] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 02:56:18,675 INFO [RS:0;jenkins-hbase4:41245] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 02:56:18,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. after waiting 0 ms 2023-07-20 02:56:18,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:56:18,675 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-20 02:56:18,676 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-20 02:56:18,677 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1478): Online Regions={1af5feb4e30a65bed26cf0457e90777d=hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d., 392fe9dfffd6a3888f97c341c2952aeb=t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb., 1588230740=hbase:meta,,1.1588230740, 300903cec0fd231ca88313bcf649662a=t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a.} 2023-07-20 02:56:18,677 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 02:56:18,677 DEBUG [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1504): Waiting on 1588230740, 1af5feb4e30a65bed26cf0457e90777d, 300903cec0fd231ca88313bcf649662a, 392fe9dfffd6a3888f97c341c2952aeb 2023-07-20 02:56:18,677 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 02:56:18,677 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 02:56:18,677 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 02:56:18,677 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 02:56:18,677 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-07-20 02:56:18,681 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/namespace/1af5feb4e30a65bed26cf0457e90777d/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-20 02:56:18,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:56:18,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1af5feb4e30a65bed26cf0457e90777d: 2023-07-20 02:56:18,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689821734364.1af5feb4e30a65bed26cf0457e90777d. 2023-07-20 02:56:18,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 392fe9dfffd6a3888f97c341c2952aeb, disabling compactions & flushes 2023-07-20 02:56:18,683 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:56:18,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:56:18,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. after waiting 0 ms 2023-07-20 02:56:18,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:56:18,683 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 392fe9dfffd6a3888f97c341c2952aeb 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-07-20 02:56:18,690 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/.tmp/info/c8d5629059ae423b842f6e49055e71a7 2023-07-20 02:56:18,702 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] wal.ProtobufLogReader(420): EOF at position 33879 2023-07-20 02:56:18,702 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.ReplicationSourceWALReader(162): Read 93 WAL entries eligible for replication 2023-07-20 02:56:18,711 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/.tmp/cf1/5f982735a3634546b33cdf9a0aeda37a 2023-07-20 02:56:18,716 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/.tmp/rep_barrier/d1d9690c2ba043dea0c678cc15f51d84 2023-07-20 02:56:18,721 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:56:18,721 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/.tmp/info/c8d5629059ae423b842f6e49055e71a7 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/info/c8d5629059ae423b842f6e49055e71a7 2023-07-20 02:56:18,728 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/info/c8d5629059ae423b842f6e49055e71a7, entries=24, sequenceid=31, filesize=7.4 K 2023-07-20 02:56:18,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/.tmp/norep/5953513717b84e76b849284893900303 2023-07-20 02:56:18,730 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/.tmp/rep_barrier/d1d9690c2ba043dea0c678cc15f51d84 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/rep_barrier/d1d9690c2ba043dea0c678cc15f51d84 2023-07-20 02:56:18,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/.tmp/cf1/5f982735a3634546b33cdf9a0aeda37a as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/cf1/5f982735a3634546b33cdf9a0aeda37a 2023-07-20 02:56:18,736 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/rep_barrier/d1d9690c2ba043dea0c678cc15f51d84, entries=2, sequenceid=31, filesize=4.9 K 2023-07-20 02:56:18,737 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 60ms, sequenceid=31, compaction requested=false 2023-07-20 02:56:18,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/cf1/5f982735a3634546b33cdf9a0aeda37a, entries=200, sequenceid=413, filesize=12.5 K 2023-07-20 02:56:18,745 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-20 02:56:18,745 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 02:56:18,746 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 02:56:18,746 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 02:56:18,746 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-20 02:56:18,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/.tmp/norep/5953513717b84e76b849284893900303 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/norep/5953513717b84e76b849284893900303 2023-07-20 02:56:18,751 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/norep/5953513717b84e76b849284893900303, entries=1, sequenceid=413, filesize=4.8 K 2023-07-20 02:56:18,752 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for 392fe9dfffd6a3888f97c341c2952aeb in 68ms, sequenceid=413, compaction requested=false 2023-07-20 02:56:18,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-07-20 02:56:18,759 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:56:18,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 392fe9dfffd6a3888f97c341c2952aeb: 2023-07-20 02:56:18,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689821739050.392fe9dfffd6a3888f97c341c2952aeb. 2023-07-20 02:56:18,760 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 300903cec0fd231ca88313bcf649662a, disabling compactions & flushes 2023-07-20 02:56:18,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:56:18,760 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:56:18,760 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. after waiting 0 ms 2023-07-20 02:56:18,760 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:56:18,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 300903cec0fd231ca88313bcf649662a 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-07-20 02:56:18,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/.tmp/cf1/ac980ec8495243eba6dea140b0cf8785 2023-07-20 02:56:18,781 WARN [RS:0;jenkins-hbase4:41245.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-20 02:56:18,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/.tmp/norep/f478ebad045c4bd7a5a4ae938a12f76c 2023-07-20 02:56:18,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/.tmp/cf1/ac980ec8495243eba6dea140b0cf8785 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/cf1/ac980ec8495243eba6dea140b0cf8785 2023-07-20 02:56:18,796 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/cf1/ac980ec8495243eba6dea140b0cf8785, entries=100, sequenceid=213, filesize=8.4 K 2023-07-20 02:56:18,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/.tmp/norep/f478ebad045c4bd7a5a4ae938a12f76c as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/norep/f478ebad045c4bd7a5a4ae938a12f76c 2023-07-20 02:56:18,802 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/norep/f478ebad045c4bd7a5a4ae938a12f76c, entries=1, sequenceid=213, filesize=4.8 K 2023-07-20 02:56:18,803 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for 300903cec0fd231ca88313bcf649662a in 43ms, sequenceid=213, compaction requested=false 2023-07-20 02:56:18,810 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/data/default/t1_syncup/300903cec0fd231ca88313bcf649662a/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-07-20 02:56:18,811 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:56:18,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 300903cec0fd231ca88313bcf649662a: 2023-07-20 02:56:18,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689821738405.300903cec0fd231ca88313bcf649662a. 2023-07-20 02:56:18,877 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41245,1689821771943; all regions closed. 2023-07-20 02:56:18,883 DEBUG [RS:0;jenkins-hbase4:41245] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs 2023-07-20 02:56:18,883 INFO [RS:0;jenkins-hbase4:41245] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41245%2C1689821771943.meta:.meta(num 1689821772465) 2023-07-20 02:56:18,888 DEBUG [RS:0;jenkins-hbase4:41245] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs 2023-07-20 02:56:18,888 INFO [RS:0;jenkins-hbase4:41245] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41245%2C1689821771943:(num 1689821772354) 2023-07-20 02:56:18,888 DEBUG [RS:0;jenkins-hbase4:41245] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:18,888 INFO [RS:0;jenkins-hbase4:41245] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:56:18,888 INFO [RS:0;jenkins-hbase4:41245] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-20 02:56:18,888 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:56:18,888 INFO [RS:0;jenkins-hbase4:41245] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-20 02:56:19,068 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410030, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:19,068 DEBUG [RS:0;jenkins-hbase4:41245] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x414b7fab to 127.0.0.1:55129 2023-07-20 02:56:19,068 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410030, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:19,069 DEBUG [RS:0;jenkins-hbase4:41245] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:19,081 WARN [RS:0;jenkins-hbase4:41245.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-07-20 02:56:19,170 DEBUG [RS:0;jenkins-hbase4:41245.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1] regionserver.HBaseInterClusterReplicationEndpoint(243): No sinks available at peer [Source for peer 1]: Interrupted while sleeping between retries 2023-07-20 02:56:19,171 INFO [RS:0;jenkins-hbase4:41245] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase4:41245.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1 terminated 2023-07-20 02:56:19,172 INFO [RS:0;jenkins-hbase4:41245] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41245 2023-07-20 02:56:19,178 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:19,178 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 02:56:19,178 ERROR [Listener at localhost/46633-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@11b80b38 rejected from java.util.concurrent.ThreadPoolExecutor@15879dd7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1374) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:603) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-07-20 02:56:19,178 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 02:56:19,179 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41245,1689821771943] 2023-07-20 02:56:19,179 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41245,1689821771943; numProcessing=1 2023-07-20 02:56:19,181 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,41245,1689821771943 already deleted, retry=false 2023-07-20 02:56:19,181 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41245,1689821771943 expired; onlineServers=0 2023-07-20 02:56:19,181 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36027,1689821771912' ***** 2023-07-20 02:56:19,181 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-20 02:56:19,182 DEBUG [M:0;jenkins-hbase4:36027] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@613c900b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:56:19,182 INFO [M:0;jenkins-hbase4:36027] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36027,1689821771912 2023-07-20 02:56:19,182 INFO [M:0;jenkins-hbase4:36027] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36027,1689821771912; all regions closed. 2023-07-20 02:56:19,182 DEBUG [M:0;jenkins-hbase4:36027] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:19,182 DEBUG [M:0;jenkins-hbase4:36027] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-20 02:56:19,182 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-20 02:56:19,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821772138] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821772138,5,FailOnTimeoutGroup] 2023-07-20 02:56:19,182 DEBUG [M:0;jenkins-hbase4:36027] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-20 02:56:19,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821772138] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821772138,5,FailOnTimeoutGroup] 2023-07-20 02:56:19,183 INFO [M:0;jenkins-hbase4:36027] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-20 02:56:19,184 INFO [M:0;jenkins-hbase4:36027] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-20 02:56:19,184 INFO [M:0;jenkins-hbase4:36027] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-20 02:56:19,184 DEBUG [M:0;jenkins-hbase4:36027] master.HMaster(1512): Stopping service threads 2023-07-20 02:56:19,184 INFO [M:0;jenkins-hbase4:36027] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-20 02:56:19,184 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-20 02:56:19,185 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 02:56:19,185 ERROR [M:0;jenkins-hbase4:36027] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-20 02:56:19,185 INFO [M:0;jenkins-hbase4:36027] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-20 02:56:19,185 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-20 02:56:19,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 02:56:19,185 DEBUG [M:0;jenkins-hbase4:36027] zookeeper.ZKUtil(398): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-20 02:56:19,185 WARN [M:0;jenkins-hbase4:36027] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-20 02:56:19,185 INFO [M:0;jenkins-hbase4:36027] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-20 02:56:19,186 INFO [M:0;jenkins-hbase4:36027] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-20 02:56:19,188 DEBUG [M:0;jenkins-hbase4:36027] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 02:56:19,188 INFO [M:0;jenkins-hbase4:36027] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:19,188 DEBUG [M:0;jenkins-hbase4:36027] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:19,188 DEBUG [M:0;jenkins-hbase4:36027] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 02:56:19,188 DEBUG [M:0;jenkins-hbase4:36027] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:19,188 INFO [M:0;jenkins-hbase4:36027] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.71 KB heapSize=36.95 KB 2023-07-20 02:56:19,198 INFO [M:0;jenkins-hbase4:36027] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.71 KB at sequenceid=218 (bloomFilter=true), to=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e4fd0fe461d6494984bbefc19de0abf6 2023-07-20 02:56:19,203 DEBUG [M:0;jenkins-hbase4:36027] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e4fd0fe461d6494984bbefc19de0abf6 as hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e4fd0fe461d6494984bbefc19de0abf6 2023-07-20 02:56:19,208 INFO [M:0;jenkins-hbase4:36027] regionserver.HStore(1080): Added hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e4fd0fe461d6494984bbefc19de0abf6, entries=10, sequenceid=218, filesize=5.8 K 2023-07-20 02:56:19,209 INFO [M:0;jenkins-hbase4:36027] regionserver.HRegion(2948): Finished flush of dataSize ~29.71 KB/30423, heapSize ~36.94 KB/37824, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 21ms, sequenceid=218, compaction requested=false 2023-07-20 02:56:19,211 INFO [M:0;jenkins-hbase4:36027] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:19,211 DEBUG [M:0;jenkins-hbase4:36027] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:56:19,214 INFO [M:0;jenkins-hbase4:36027] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-20 02:56:19,214 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:56:19,214 INFO [M:0;jenkins-hbase4:36027] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36027 2023-07-20 02:56:19,216 DEBUG [M:0;jenkins-hbase4:36027] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,36027,1689821771912 already deleted, retry=false 2023-07-20 02:56:19,280 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:19,280 INFO [RS:0;jenkins-hbase4:41245] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41245,1689821771943; zookeeper connection closed. 2023-07-20 02:56:19,280 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41245-0x10180a7ca41002c, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:19,281 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@142e76a5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@142e76a5 2023-07-20 02:56:19,281 INFO [Listener at localhost/46633] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-20 02:56:19,380 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:19,380 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:36027-0x10180a7ca41002b, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:19,380 INFO [M:0;jenkins-hbase4:36027] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36027,1689821771912; zookeeper connection closed. 2023-07-20 02:56:19,383 INFO [Listener at localhost/46633] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 02:56:19,383 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:19,383 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:19,384 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:56:19,384 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:19,384 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:56:19,384 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:56:19,384 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33365 2023-07-20 02:56:19,385 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:56:19,386 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:56:19,387 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33365 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:56:19,389 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:333650x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:56:19,390 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33365-0x10180a7ca410033 connected 2023-07-20 02:56:19,393 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:56:19,393 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:56:19,393 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 02:56:19,394 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33365 2023-07-20 02:56:19,394 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33365 2023-07-20 02:56:19,394 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33365 2023-07-20 02:56:19,394 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33365 2023-07-20 02:56:19,395 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33365 2023-07-20 02:56:19,395 INFO [Listener at localhost/46633] master.HMaster(444): hbase.rootdir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16, hbase.cluster.distributed=false 2023-07-20 02:56:19,395 DEBUG [pool-640-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-20 02:56:19,407 INFO [Listener at localhost/46633] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 02:56:19,407 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:19,407 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:19,407 INFO [Listener at localhost/46633] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 02:56:19,408 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 02:56:19,408 INFO [Listener at localhost/46633] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 02:56:19,408 INFO [Listener at localhost/46633] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 02:56:19,408 INFO [Listener at localhost/46633] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41413 2023-07-20 02:56:19,409 INFO [Listener at localhost/46633] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 02:56:19,410 DEBUG [Listener at localhost/46633] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 02:56:19,410 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:56:19,411 INFO [Listener at localhost/46633] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:56:19,412 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41413 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:56:19,414 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:414130x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:56:19,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41413-0x10180a7ca410034 connected 2023-07-20 02:56:19,416 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:56:19,416 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:56:19,417 DEBUG [Listener at localhost/46633] zookeeper.ZKUtil(164): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 02:56:19,417 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41413 2023-07-20 02:56:19,417 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41413 2023-07-20 02:56:19,417 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41413 2023-07-20 02:56:19,418 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41413 2023-07-20 02:56:19,418 DEBUG [Listener at localhost/46633] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41413 2023-07-20 02:56:19,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,33365,1689821779383 2023-07-20 02:56:19,420 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 02:56:19,421 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,33365,1689821779383 2023-07-20 02:56:19,422 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 02:56:19,422 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:56:19,422 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 02:56:19,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 02:56:19,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,33365,1689821779383 from backup master directory 2023-07-20 02:56:19,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 02:56:19,427 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,33365,1689821779383 2023-07-20 02:56:19,427 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:56:19,427 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 02:56:19,427 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33365,1689821779383 2023-07-20 02:56:19,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 02:56:19,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x0d31ea56 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:19,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f88c8d6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:19,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 02:56:19,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-20 02:56:19,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:56:19,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,40859,1689821745131 to hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,40859,1689821745131-dead as it is dead 2023-07-20 02:56:19,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,40859,1689821745131-dead/jenkins-hbase4.apache.org%2C40859%2C1689821745131.1689821745272 2023-07-20 02:56:19,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,40859,1689821745131-dead/jenkins-hbase4.apache.org%2C40859%2C1689821745131.1689821745272 after 0ms 2023-07-20 02:56:19,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,40859,1689821745131-dead/jenkins-hbase4.apache.org%2C40859%2C1689821745131.1689821745272 to hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40859%2C1689821745131.1689821745272 2023-07-20 02:56:19,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,40859,1689821745131-dead 2023-07-20 02:56:19,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,33365,1689821779383 2023-07-20 02:56:19,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33365%2C1689821779383, suffix=, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,33365,1689821779383, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/oldWALs, maxLogs=10 2023-07-20 02:56:19,495 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK] 2023-07-20 02:56:19,496 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK] 2023-07-20 02:56:19,496 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK] 2023-07-20 02:56:19,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/WALs/jenkins-hbase4.apache.org,33365,1689821779383/jenkins-hbase4.apache.org%2C33365%2C1689821779383.1689821779474 2023-07-20 02:56:19,498 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK], DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK], DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK]] 2023-07-20 02:56:19,498 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:56:19,498 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:56:19,499 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:56:19,499 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:56:19,500 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:56:19,501 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-20 02:56:19,501 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-20 02:56:19,507 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7538bca05e2a4ba7834e6fa33e0cf942 2023-07-20 02:56:19,510 INFO [StoreFileOpener-proc-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for adfd5ca96e9945a5b803a00e00421eac 2023-07-20 02:56:19,510 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/adfd5ca96e9945a5b803a00e00421eac 2023-07-20 02:56:19,511 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:19,511 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-20 02:56:19,511 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40859%2C1689821745131.1689821745272 2023-07-20 02:56:19,516 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 139, firstSequenceIdInLog=120, maxSequenceIdInLog=240, path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40859%2C1689821745131.1689821745272 2023-07-20 02:56:19,517 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40859%2C1689821745131.1689821745272 2023-07-20 02:56:19,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 02:56:19,522 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/240.seqid, newMaxSeqId=240, maxSeqId=118 2023-07-20 02:56:19,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=241; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65623975, jitterRate=-0.022126570343971252}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 02:56:19,523 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:56:19,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-20 02:56:19,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-20 02:56:19,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-20 02:56:19,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-20 02:56:19,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-20 02:56:19,531 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-20 02:56:19,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-20 02:56:19,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-20 02:56:19,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-20 02:56:19,533 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-20 02:56:19,533 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-20 02:56:19,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-20 02:56:19,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-20 02:56:19,536 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-20 02:56:19,537 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38383,1689821745167, table=hbase:meta, region=1588230740 2023-07-20 02:56:19,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-07-20 02:56:19,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,38383,1689821745167 already deleted, retry=false 2023-07-20 02:56:19,540 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,38383,1689821745167 on jenkins-hbase4.apache.org,33365,1689821779383 2023-07-20 02:56:19,541 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,38383,1689821745167, splitWal=true, meta=true 2023-07-20 02:56:19,541 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase4.apache.org,38383,1689821745167 (carryingMeta=true) jenkins-hbase4.apache.org,38383,1689821745167/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@3d81fab[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 02:56:19,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-20 02:56:19,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-20 02:56:19,553 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-20 02:56:19,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-20 02:56:19,553 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-20 02:56:19,554 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-20 02:56:19,554 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-20 02:56:19,555 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-20 02:56:19,557 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 02:56:19,557 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 02:56:19,557 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:56:19,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33365,1689821779383, sessionid=0x10180a7ca410033, setting cluster-up flag (Was=false) 2023-07-20 02:56:19,563 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-20 02:56:19,564 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33365,1689821779383 2023-07-20 02:56:19,567 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-20 02:56:19,568 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33365,1689821779383 2023-07-20 02:56:19,569 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/.hbase-snapshot/.tmp 2023-07-20 02:56:19,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:56:19,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:56:19,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:56:19,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 02:56:19,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-20 02:56:19,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:56:19,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689821809574 2023-07-20 02:56:19,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-20 02:56:19,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-20 02:56:19,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-20 02:56:19,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-20 02:56:19,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-20 02:56:19,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-20 02:56:19,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:19,576 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-20 02:56:19,577 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38383,1689821745167; numProcessing=1 2023-07-20 02:56:19,577 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-20 02:56:19,577 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38383,1689821745167, splitWal=true, meta=true 2023-07-20 02:56:19,577 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-20 02:56:19,578 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-20 02:56:19,578 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-20 02:56:19,578 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821779578,5,FailOnTimeoutGroup] 2023-07-20 02:56:19,578 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38383,1689821745167, splitWal=true, meta=true, isMeta: true 2023-07-20 02:56:19,579 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821779578,5,FailOnTimeoutGroup] 2023-07-20 02:56:19,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:19,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-20 02:56:19,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:19,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:19,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689821779579, completionTime=-1 2023-07-20 02:56:19,579 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-20 02:56:19,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 02:56:19,580 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167-splitting 2023-07-20 02:56:19,581 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167-splitting dir is empty, no logs to split. 2023-07-20 02:56:19,581 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,38383,1689821745167 WAL count=0, meta=true 2023-07-20 02:56:19,583 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167-splitting dir is empty, no logs to split. 2023-07-20 02:56:19,583 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,38383,1689821745167 WAL count=0, meta=true 2023-07-20 02:56:19,583 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,38383,1689821745167 WAL splitting is done? wals=0, meta=true 2023-07-20 02:56:19,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-20 02:56:19,585 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-20 02:56:19,587 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:56:19,619 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(951): ClusterId : 69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:56:19,620 DEBUG [RS:0;jenkins-hbase4:41413] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 02:56:19,621 DEBUG [RS:0;jenkins-hbase4:41413] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 02:56:19,621 DEBUG [RS:0;jenkins-hbase4:41413] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 02:56:19,629 DEBUG [RS:0;jenkins-hbase4:41413] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 02:56:19,631 DEBUG [RS:0;jenkins-hbase4:41413] zookeeper.ReadOnlyZKClient(139): Connect 0x412e8259 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:19,635 DEBUG [RS:0;jenkins-hbase4:41413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4fe9640c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:19,635 DEBUG [RS:0;jenkins-hbase4:41413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4bd6e929, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:56:19,644 DEBUG [RS:0;jenkins-hbase4:41413] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:41413 2023-07-20 02:56:19,644 INFO [RS:0;jenkins-hbase4:41413] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 02:56:19,644 INFO [RS:0;jenkins-hbase4:41413] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 02:56:19,644 DEBUG [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 02:56:19,644 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33365,1689821779383 with isa=jenkins-hbase4.apache.org/172.31.14.131:41413, startcode=1689821779407 2023-07-20 02:56:19,645 DEBUG [RS:0;jenkins-hbase4:41413] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 02:56:19,646 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40457, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 02:56:19,647 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33365] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:19,648 DEBUG [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16 2023-07-20 02:56:19,648 DEBUG [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40517 2023-07-20 02:56:19,648 DEBUG [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 02:56:19,649 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca41002a, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:19,649 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:19,649 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:56:19,649 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410027, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:19,649 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:56:19,650 DEBUG [RS:0;jenkins-hbase4:41413] zookeeper.ZKUtil(162): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:19,651 WARN [RS:0;jenkins-hbase4:41413] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 02:56:19,651 INFO [RS:0;jenkins-hbase4:41413] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:56:19,651 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41413,1689821779407] 2023-07-20 02:56:19,651 DEBUG [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:19,660 DEBUG [RS:0;jenkins-hbase4:41413] zookeeper.ZKUtil(162): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:19,661 DEBUG [RS:0;jenkins-hbase4:41413] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:56:19,661 INFO [RS:0;jenkins-hbase4:41413] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 02:56:19,663 INFO [RS:0;jenkins-hbase4:41413] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 02:56:19,664 INFO [RS:0;jenkins-hbase4:41413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 02:56:19,664 INFO [RS:0;jenkins-hbase4:41413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:19,664 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 02:56:19,666 INFO [RS:0;jenkins-hbase4:41413] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:19,666 DEBUG [RS:0;jenkins-hbase4:41413] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,666 DEBUG [RS:0;jenkins-hbase4:41413] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,666 DEBUG [RS:0;jenkins-hbase4:41413] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,666 DEBUG [RS:0;jenkins-hbase4:41413] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,666 DEBUG [RS:0;jenkins-hbase4:41413] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,666 DEBUG [RS:0;jenkins-hbase4:41413] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 02:56:19,666 DEBUG [RS:0;jenkins-hbase4:41413] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,666 DEBUG [RS:0;jenkins-hbase4:41413] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,666 DEBUG [RS:0;jenkins-hbase4:41413] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,666 DEBUG [RS:0;jenkins-hbase4:41413] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 02:56:19,668 INFO [RS:0;jenkins-hbase4:41413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:19,668 INFO [RS:0;jenkins-hbase4:41413] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:19,668 INFO [RS:0;jenkins-hbase4:41413] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:19,680 INFO [RS:0;jenkins-hbase4:41413] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 02:56:19,680 INFO [RS:0;jenkins-hbase4:41413] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41413,1689821779407-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:19,680 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=101ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 02:56:19,691 INFO [RS:0;jenkins-hbase4:41413] regionserver.Replication(203): jenkins-hbase4.apache.org,41413,1689821779407 started 2023-07-20 02:56:19,691 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41413,1689821779407, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41413, sessionid=0x10180a7ca410034 2023-07-20 02:56:19,691 DEBUG [RS:0;jenkins-hbase4:41413] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 02:56:19,691 DEBUG [RS:0;jenkins-hbase4:41413] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:19,691 DEBUG [RS:0;jenkins-hbase4:41413] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41413,1689821779407' 2023-07-20 02:56:19,691 DEBUG [RS:0;jenkins-hbase4:41413] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 02:56:19,692 DEBUG [RS:0;jenkins-hbase4:41413] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 02:56:19,692 DEBUG [RS:0;jenkins-hbase4:41413] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 02:56:19,692 DEBUG [RS:0;jenkins-hbase4:41413] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 02:56:19,692 DEBUG [RS:0;jenkins-hbase4:41413] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:19,692 DEBUG [RS:0;jenkins-hbase4:41413] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41413,1689821779407' 2023-07-20 02:56:19,692 DEBUG [RS:0;jenkins-hbase4:41413] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 02:56:19,692 DEBUG [RS:0;jenkins-hbase4:41413] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 02:56:19,693 DEBUG [RS:0;jenkins-hbase4:41413] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 02:56:19,693 INFO [RS:0;jenkins-hbase4:41413] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 02:56:19,693 INFO [RS:0;jenkins-hbase4:41413] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 02:56:19,737 DEBUG [jenkins-hbase4:33365] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 02:56:19,738 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41413,1689821779407, state=OPENING 2023-07-20 02:56:19,740 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-20 02:56:19,740 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 02:56:19,740 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41413,1689821779407}] 2023-07-20 02:56:19,795 INFO [RS:0;jenkins-hbase4:41413] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41413%2C1689821779407, suffix=, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,41413,1689821779407, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs, maxLogs=10 2023-07-20 02:56:19,812 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK] 2023-07-20 02:56:19,812 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK] 2023-07-20 02:56:19,812 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK] 2023-07-20 02:56:19,814 INFO [RS:0;jenkins-hbase4:41413] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,41413,1689821779407/jenkins-hbase4.apache.org%2C41413%2C1689821779407.1689821779795 2023-07-20 02:56:19,814 DEBUG [RS:0;jenkins-hbase4:41413] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK], DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK], DatanodeInfoWithStorage[127.0.0.1:34649,DS-48b44248-b692-4d7a-b9b0-5c8a38a185fb,DISK]] 2023-07-20 02:56:19,895 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:19,896 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:56:19,897 INFO [RS-EventLoopGroup-17-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41102, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:56:19,900 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-20 02:56:19,900 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:56:19,902 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41413%2C1689821779407.meta, suffix=.meta, logDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,41413,1689821779407, archiveDir=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs, maxLogs=10 2023-07-20 02:56:19,916 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK] 2023-07-20 02:56:19,916 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK] 2023-07-20 02:56:19,916 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK] 2023-07-20 02:56:19,918 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,41413,1689821779407/jenkins-hbase4.apache.org%2C41413%2C1689821779407.meta.1689821779903.meta 2023-07-20 02:56:19,919 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34619,DS-87be6ac9-bf85-4b5b-85aa-76790df4118c,DISK], DatanodeInfoWithStorage[127.0.0.1:38071,DS-b74a8495-a7ff-448d-bd64-a7d9320b197d,DISK], DatanodeInfoWithStorage[127.0.0.1:42001,DS-e7487476-2418-4cf7-be23-93b7a8d3eb0a,DISK]] 2023-07-20 02:56:19,919 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:56:19,919 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-20 02:56:19,919 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-20 02:56:19,919 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-20 02:56:19,919 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-20 02:56:19,919 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:56:19,920 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-20 02:56:19,920 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-20 02:56:19,921 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 02:56:19,922 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info 2023-07-20 02:56:19,922 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info 2023-07-20 02:56:19,922 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 02:56:19,928 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info/441251dfb1734be1aa02887ae7be7817 2023-07-20 02:56:19,932 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info/96735ff5a6a9432c98af6ae4fc0b5b78 2023-07-20 02:56:19,932 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:19,932 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 02:56:19,933 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:56:19,933 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/rep_barrier 2023-07-20 02:56:19,933 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 02:56:19,933 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:19,933 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 02:56:19,934 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table 2023-07-20 02:56:19,934 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table 2023-07-20 02:56:19,935 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 02:56:19,940 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/table/838ab0139f504a0b99265e300d89842c 2023-07-20 02:56:19,940 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:19,940 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740 2023-07-20 02:56:19,941 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740 2023-07-20 02:56:19,944 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 02:56:19,945 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 02:56:19,946 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64539837, jitterRate=-0.03828148543834686}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 02:56:19,946 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 02:56:19,946 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1689821779895 2023-07-20 02:56:19,951 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-20 02:56:19,951 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-20 02:56:19,952 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41413,1689821779407, state=OPEN 2023-07-20 02:56:19,954 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-20 02:56:19,954 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 02:56:19,956 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-20 02:56:19,956 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41413,1689821779407 in 214 msec 2023-07-20 02:56:19,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-07-20 02:56:19,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 372 msec 2023-07-20 02:56:21,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1604ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-20 02:56:22,503 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 02:56:22,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3110ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3009ms 2023-07-20 02:56:24,092 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4513ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-20 02:56:24,092 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-20 02:56:24,096 DEBUG [hconnection-0x10c89c03-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:56:24,097 INFO [RS-EventLoopGroup-17-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53598, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:56:24,106 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=956fe6fd22cc055b7d5c09ea7e2fd4a7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38383,1689821745167, regionLocation=jenkins-hbase4.apache.org,38383,1689821745167, openSeqNum=10 2023-07-20 02:56:24,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=80b6625deaec89ca31a4c1b65e4a6544, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38383,1689821745167, regionLocation=jenkins-hbase4.apache.org,38383,1689821745167, openSeqNum=11 2023-07-20 02:56:24,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=6908f43230333d97246933b2af61e211, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38383,1689821745167, regionLocation=jenkins-hbase4.apache.org,38383,1689821745167, openSeqNum=12 2023-07-20 02:56:24,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-20 02:56:24,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689821844107 2023-07-20 02:56:24,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689821904107 2023-07-20 02:56:24,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 15 msec 2023-07-20 02:56:24,126 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,38383,1689821745167 had 4 regions 2023-07-20 02:56:24,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33365,1689821779383-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:24,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33365,1689821779383-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:24,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33365,1689821779383-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:24,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33365, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:24,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-20 02:56:24,127 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. is NOT online; state={956fe6fd22cc055b7d5c09ea7e2fd4a7 state=OPEN, ts=1689821784107, server=jenkins-hbase4.apache.org,38383,1689821745167}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-20 02:56:24,129 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38383,1689821745167, splitWal=true, meta=true, isMeta: false 2023-07-20 02:56:24,132 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,38383,1689821745167/hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7., unknown_server=jenkins-hbase4.apache.org,38383,1689821745167/t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544., unknown_server=jenkins-hbase4.apache.org,38383,1689821745167/t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:24,132 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167-splitting dir is empty, no logs to split. 2023-07-20 02:56:24,132 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,38383,1689821745167 WAL count=0, meta=false 2023-07-20 02:56:24,135 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/WALs/jenkins-hbase4.apache.org,38383,1689821745167-splitting dir is empty, no logs to split. 2023-07-20 02:56:24,135 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,38383,1689821745167 WAL count=0, meta=false 2023-07-20 02:56:24,135 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,38383,1689821745167 WAL splitting is done? wals=0, meta=false 2023-07-20 02:56:24,139 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN}] 2023-07-20 02:56:24,140 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN 2023-07-20 02:56:24,140 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN 2023-07-20 02:56:24,140 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN 2023-07-20 02:56:24,141 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:56:24,141 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:56:24,141 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 02:56:24,141 DEBUG [jenkins-hbase4:33365] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 02:56:24,145 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=80b6625deaec89ca31a4c1b65e4a6544, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:24,145 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=6908f43230333d97246933b2af61e211, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:24,145 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821784145"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821784145"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821784145"}]},"ts":"1689821784145"} 2023-07-20 02:56:24,145 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821784145"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821784145"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821784145"}]},"ts":"1689821784145"} 2023-07-20 02:56:24,145 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=956fe6fd22cc055b7d5c09ea7e2fd4a7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:24,146 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821784145"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689821784145"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689821784145"}]},"ts":"1689821784145"} 2023-07-20 02:56:24,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 6908f43230333d97246933b2af61e211, server=jenkins-hbase4.apache.org,41413,1689821779407}] 2023-07-20 02:56:24,153 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=30, state=RUNNABLE; OpenRegionProcedure 80b6625deaec89ca31a4c1b65e4a6544, server=jenkins-hbase4.apache.org,41413,1689821779407}] 2023-07-20 02:56:24,158 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=31, state=RUNNABLE; OpenRegionProcedure 956fe6fd22cc055b7d5c09ea7e2fd4a7, server=jenkins-hbase4.apache.org,41413,1689821779407}] 2023-07-20 02:56:24,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:24,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 80b6625deaec89ca31a4c1b65e4a6544, NAME => 't1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:56:24,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:56:24,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:56:24,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:56:24,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:56:24,310 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:56:24,312 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1 2023-07-20 02:56:24,312 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1 2023-07-20 02:56:24,312 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80b6625deaec89ca31a4c1b65e4a6544 columnFamilyName cf1 2023-07-20 02:56:24,314 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:24,314 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 956fe6fd22cc055b7d5c09ea7e2fd4a7, NAME => 'hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:56:24,314 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:24,314 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:56:24,314 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:24,314 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:24,317 INFO [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:24,318 DEBUG [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/info 2023-07-20 02:56:24,318 DEBUG [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/info 2023-07-20 02:56:24,319 INFO [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 956fe6fd22cc055b7d5c09ea7e2fd4a7 columnFamilyName info 2023-07-20 02:56:24,320 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 12d666253300492d8b93c8e4058ffad7 2023-07-20 02:56:24,320 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1/12d666253300492d8b93c8e4058ffad7 2023-07-20 02:56:24,325 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1/db3cbecb4ef74734b4315fa2f2bf0702 2023-07-20 02:56:24,325 DEBUG [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/info/ba087245ad7f4c18b071b799180f415c 2023-07-20 02:56:24,325 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(310): Store=80b6625deaec89ca31a4c1b65e4a6544/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:24,325 INFO [StoreOpener-956fe6fd22cc055b7d5c09ea7e2fd4a7-1] regionserver.HStore(310): Store=956fe6fd22cc055b7d5c09ea7e2fd4a7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:24,325 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:56:24,326 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:24,327 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/norep 2023-07-20 02:56:24,327 DEBUG [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/norep 2023-07-20 02:56:24,327 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80b6625deaec89ca31a4c1b65e4a6544 columnFamilyName norep 2023-07-20 02:56:24,327 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:24,328 INFO [StoreOpener-80b6625deaec89ca31a4c1b65e4a6544-1] regionserver.HStore(310): Store=80b6625deaec89ca31a4c1b65e4a6544/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:24,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:56:24,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:56:24,331 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:24,331 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 956fe6fd22cc055b7d5c09ea7e2fd4a7; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64638836, jitterRate=-0.03680628538131714}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 02:56:24,332 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 956fe6fd22cc055b7d5c09ea7e2fd4a7: 2023-07-20 02:56:24,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 02:56:24,332 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7., pid=35, masterSystemTime=1689821784304 2023-07-20 02:56:24,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:56:24,335 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:24,335 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:24,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 80b6625deaec89ca31a4c1b65e4a6544; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72877385, jitterRate=0.08595766127109528}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 02:56:24,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 80b6625deaec89ca31a4c1b65e4a6544: 2023-07-20 02:56:24,336 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=956fe6fd22cc055b7d5c09ea7e2fd4a7, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:24,336 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689821784335"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821784335"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821784335"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821784335"}]},"ts":"1689821784335"} 2023-07-20 02:56:24,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544., pid=34, masterSystemTime=1689821784304 2023-07-20 02:56:24,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:24,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:24,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:24,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6908f43230333d97246933b2af61e211, NAME => 't2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.', STARTKEY => '', ENDKEY => ''} 2023-07-20 02:56:24,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 6908f43230333d97246933b2af61e211 2023-07-20 02:56:24,338 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=80b6625deaec89ca31a4c1b65e4a6544, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:24,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 02:56:24,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6908f43230333d97246933b2af61e211 2023-07-20 02:56:24,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6908f43230333d97246933b2af61e211 2023-07-20 02:56:24,339 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=31 2023-07-20 02:56:24,339 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821784338"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821784338"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821784338"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821784338"}]},"ts":"1689821784338"} 2023-07-20 02:56:24,339 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=31, state=SUCCESS; OpenRegionProcedure 956fe6fd22cc055b7d5c09ea7e2fd4a7, server=jenkins-hbase4.apache.org,41413,1689821779407 in 182 msec 2023-07-20 02:56:24,340 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6908f43230333d97246933b2af61e211 2023-07-20 02:56:24,340 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=956fe6fd22cc055b7d5c09ea7e2fd4a7, ASSIGN in 203 msec 2023-07-20 02:56:24,341 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1 2023-07-20 02:56:24,341 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1 2023-07-20 02:56:24,342 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=30 2023-07-20 02:56:24,342 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=30, state=SUCCESS; OpenRegionProcedure 80b6625deaec89ca31a4c1b65e4a6544, server=jenkins-hbase4.apache.org,41413,1689821779407 in 187 msec 2023-07-20 02:56:24,342 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6908f43230333d97246933b2af61e211 columnFamilyName cf1 2023-07-20 02:56:24,343 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=80b6625deaec89ca31a4c1b65e4a6544, ASSIGN in 206 msec 2023-07-20 02:56:24,348 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1/a2587aab5db342319f7ea6dadbb7933e 2023-07-20 02:56:24,352 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ccb9665ed0cc48858e87b550c7de02a8 2023-07-20 02:56:24,352 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(539): loaded hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1/ccb9665ed0cc48858e87b550c7de02a8 2023-07-20 02:56:24,353 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(310): Store=6908f43230333d97246933b2af61e211/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:24,353 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 6908f43230333d97246933b2af61e211 2023-07-20 02:56:24,354 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/norep 2023-07-20 02:56:24,354 DEBUG [StoreOpener-6908f43230333d97246933b2af61e211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/norep 2023-07-20 02:56:24,354 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6908f43230333d97246933b2af61e211 columnFamilyName norep 2023-07-20 02:56:24,355 INFO [StoreOpener-6908f43230333d97246933b2af61e211-1] regionserver.HStore(310): Store=6908f43230333d97246933b2af61e211/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 02:56:24,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211 2023-07-20 02:56:24,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211 2023-07-20 02:56:24,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 02:56:24,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6908f43230333d97246933b2af61e211 2023-07-20 02:56:24,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6908f43230333d97246933b2af61e211; next sequenceid=19; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63031596, jitterRate=-0.060756027698516846}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 02:56:24,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6908f43230333d97246933b2af61e211: 2023-07-20 02:56:24,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211., pid=33, masterSystemTime=1689821784304 2023-07-20 02:56:24,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:24,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:24,365 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=6908f43230333d97246933b2af61e211, regionState=OPEN, openSeqNum=19, regionLocation=jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:24,365 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689821784365"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689821784365"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689821784365"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689821784365"}]},"ts":"1689821784365"} 2023-07-20 02:56:24,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-07-20 02:56:24,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 6908f43230333d97246933b2af61e211, server=jenkins-hbase4.apache.org,41413,1689821779407 in 216 msec 2023-07-20 02:56:24,370 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=27 2023-07-20 02:56:24,370 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,38383,1689821745167 after splitting done 2023-07-20 02:56:24,370 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=6908f43230333d97246933b2af61e211, ASSIGN in 232 msec 2023-07-20 02:56:24,371 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,38383,1689821745167 from processing; numProcessing=0 2023-07-20 02:56:24,372 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,38383,1689821745167, splitWal=true, meta=true in 4.8310 sec 2023-07-20 02:56:24,691 INFO [jenkins-hbase4:41413Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:56:25,128 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-20 02:56:25,145 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-20 02:56:25,153 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-20 02:56:25,153 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.726sec 2023-07-20 02:56:25,153 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-20 02:56:25,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-20 02:56:25,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-20 02:56:25,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33365,1689821779383-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-20 02:56:25,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33365,1689821779383-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-20 02:56:25,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-20 02:56:25,224 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(139): Connect 0x02689d38 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:25,228 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@fc1791e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:25,230 DEBUG [hconnection-0x7998881d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:56:25,231 INFO [RS-EventLoopGroup-17-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53612, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:56:25,233 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-20 02:56:25,234 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x02689d38 to 127.0.0.1:55129 2023-07-20 02:56:25,234 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:25,235 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:33365 after: jenkins-hbase4.apache.org:33365 2023-07-20 02:56:25,235 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(139): Connect 0x69f8714c to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:25,241 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@541dd40e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:25,662 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-20 02:56:25,662 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-20 02:56:25,662 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-20 02:56:25,663 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-20 02:56:25,743 DEBUG [hconnection-0x6ebed27e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:56:25,744 INFO [RS-EventLoopGroup-17-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53624, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 02:56:25,795 INFO [Listener at localhost/46633] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689821785795 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:56:25,797 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): syncupReplication16898217857950x0, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:56:25,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689821785795-0x10180a7ca410039 connected Start Replication Server start 2023-07-20 02:56:25,802 INFO [Listener at localhost/46633] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-20 02:56:25,817 INFO [Listener at localhost/46633] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 02:56:25,820 DEBUG [Listener at localhost/46633] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 02:56:25,834 INFO [Listener at localhost/46633] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-20 02:56:25,837 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,41245,1689821771943] other RSs: [] 2023-07-20 02:56:25,859 DEBUG [Listener at localhost/46633.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x5e40ad8e to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:25,863 DEBUG [Listener at localhost/46633.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@75af402c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:25,864 INFO [Listener at localhost/46633.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:56:25,868 DEBUG [Listener at localhost/46633.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:56:25,870 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10180a7ca41003b connected 2023-07-20 02:56:25,870 INFO [Listener at localhost/46633.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=bbd9fa59-2213-4359-9606-c616f1cd40eb to cluster=69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:56:29,576 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,41245,1689821771943/1's WALs to 1689821785802.syncuptool.replication.org,1234,1 2023-07-20 02:56:29,578 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C41245%2C1689821771943.1689821772354 with data PBUF\x08\x8A\x09 2023-07-20 02:56:29,580 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,41245,1689821771943/1's WALs to 1689821785802.syncuptool.replication.org,1234,1 2023-07-20 02:56:29,691 INFO [jenkins-hbase4:41413Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:56:29,812 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 02:56:31,583 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41245,1689821771943] 2023-07-20 02:56:31,583 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41245,1689821771943] 2023-07-20 02:56:31,594 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41245,1689821771943] 2023-07-20 02:56:31,594 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,41245,1689821771943, ReplicationSource: 1, currentBandwidth=0 2023-07-20 02:56:31,594 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:31,615 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943] zookeeper.ReadOnlyZKClient(139): Connect 0x0039fd2f to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:31,619 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3afc824, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:31,619 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:55129 2023-07-20 02:56:31,621 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 02:56:31,622 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10180a7ca41003d connected 2023-07-20 02:56:31,622 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,41245,1689821771943 (queues=1) is replicating from cluster=bbd9fa59-2213-4359-9606-c616f1cd40eb to cluster=69eaa123-7a27-404c-9839-15690c09154d 2023-07-20 02:56:31,623 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C41245%2C1689821771943 2023-07-20 02:56:31,626 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,41245,1689821771943, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 02:56:31,626 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C41245%2C1689821771943 2023-07-20 02:56:31,636 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs/jenkins-hbase4.apache.org%2C41245%2C1689821771943.1689821772354 2023-07-20 02:56:31,636 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.ReplicationSourceWALReader(162): Read 300 WAL entries eligible for replication 2023-07-20 02:56:31,636 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-20 02:56:31,638 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,41245,1689821771943 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/300903cec0fd231ca88313bcf649662a/112=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/113=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/114=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/115=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/116=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/117=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/118=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/119=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/120=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/121=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/122=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/123=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/124=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/125=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/126=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/127=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/128=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/129=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/130=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/131=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/132=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/133=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/134=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/135=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/136=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/137=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/138=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/139=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/140=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/141=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/142=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/143=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/144=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/145=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/146=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/147=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/148=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/149=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/150=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/151=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/152=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/153=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/154=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/155=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/156=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/157=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/158=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/159=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/160=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/161=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/162=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/163=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/164=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/165=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/166=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/167=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/168=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/169=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/170=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/171=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/172=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/173=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/174=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/175=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/176=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/177=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/178=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/179=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/180=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/181=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/182=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/183=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/184=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/185=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/186=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/187=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/188=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/189=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/190=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/191=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/192=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/193=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/194=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/195=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/196=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/197=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/198=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/199=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/200=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/201=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/202=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/203=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/204=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/205=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/206=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/207=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/208=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/209=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/210=[#edits: 1 = ],208}, {t1_syncup/300903cec0fd231ca88313bcf649662a/211=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/212=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/213=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/214=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/215=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/216=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/217=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/218=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/219=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/220=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/221=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/222=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/223=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/224=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/225=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/226=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/227=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/228=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/229=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/230=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/231=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/232=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/233=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/234=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/235=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/236=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/237=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/238=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/239=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/240=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/241=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/242=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/243=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/244=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/245=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/246=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/247=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/248=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/249=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/250=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/251=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/252=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/253=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/254=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/255=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/256=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/257=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/258=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/259=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/260=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/261=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/262=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/263=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/264=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/265=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/266=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/267=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/268=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/269=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/270=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/271=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/272=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/273=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/274=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/275=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/276=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/277=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/278=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/279=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/280=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/281=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/282=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/283=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/284=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/285=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/286=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/287=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/288=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/289=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/290=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/291=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/292=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/293=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/294=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/295=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/296=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/297=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/298=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/299=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/300=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/301=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/302=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/303=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/304=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/305=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/306=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/307=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/308=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/309=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/310=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/311=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/312=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/313=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/314=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/315=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/316=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/317=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/318=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/319=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/320=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/321=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/322=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/323=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/324=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/325=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/326=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/327=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/328=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/329=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/330=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/331=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/332=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/333=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/334=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/335=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/336=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/337=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/338=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/339=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/340=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/341=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/342=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/343=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/344=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/345=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/346=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/347=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/348=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/349=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/350=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/351=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/352=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/353=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/354=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/355=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/356=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/357=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/358=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/359=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/360=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/361=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/362=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/363=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/364=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/365=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/366=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/367=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/368=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/369=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/370=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/371=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/372=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/373=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/374=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/375=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/376=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/377=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/378=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/379=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/380=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/381=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/382=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/383=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/384=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/385=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/386=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/387=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/388=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/389=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/390=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/391=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/392=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/393=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/394=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/395=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/396=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/397=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/398=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/399=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/400=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/401=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/402=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/403=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/404=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/405=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/406=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/407=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/408=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/409=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/410=[#edits: 1 = ],208}, {t2_syncup/392fe9dfffd6a3888f97c341c2952aeb/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:37763/user/jenkins/test-data/4237d7a8-d6ce-65ae-ca50-e199cb617d61/oldWALs/jenkins-hbase4.apache.org%2C41245%2C1689821771943.1689821772354, lastWalPosition=35352, nbRowKeys=300, nbHFiles=0, heapSize=62400, lastSeqIds={}, endOfFile=true,usedBufferSize=62400] 2023-07-20 02:56:31,642 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 02:56:31,644 INFO [RS-EventLoopGroup-17-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34056, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 02:56:31,648 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41413] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 02:56:31,649 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41413] zookeeper.ReadOnlyZKClient(139): Connect 0x4cf1c817 to 127.0.0.1:55129 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 02:56:31,652 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@42363ff9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 02:56:31,653 DEBUG [hconnection-0x4609a367-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 02:56:31,655 INFO [RS-EventLoopGroup-17-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34058, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-07-20 02:56:31,673 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41413] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 02:56:31,673 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41245,1689821771943] 2023-07-20 02:56:31,676 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C41245%2C1689821771943.1689821772354] 2023-07-20 02:56:31,678 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,41245,1689821771943 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 02:56:31,678 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C41245%2C1689821771943 of peer 1-jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:31,778 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:31,780 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689821785795-0x10180a7ca410039, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689821785802.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,41245,1689821771943 2023-07-20 02:56:31,780 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943.replicationSource.shipperjenkins-hbase4.apache.org%2C41245%2C1689821771943,1-jenkins-hbase4.apache.org,41245,1689821771943] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,41245,1689821771943 with the following stats: Total replicated edits: 300, current progress: 2023-07-20 02:56:34,691 INFO [jenkins-hbase4:41413Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 02:56:34,691 INFO [jenkins-hbase4:41413Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-07-20 02:56:35,837 INFO [Listener at localhost/46633] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-20 02:56:35,941 DEBUG [Listener at localhost/46633.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca41003b, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:35,941 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5e40ad8e to 127.0.0.1:55129 2023-07-20 02:56:35,942 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:36,145 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689821785795-0x10180a7ca410039, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:36,145 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689821785795-0x10180a7ca410039, quorum=127.0.0.1:55129, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:36,153 INFO [Listener at localhost/46633] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-07-20 02:56:36,154 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-20 02:56:36,154 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x69f8714c to 127.0.0.1:55129 2023-07-20 02:56:36,154 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:36,154 DEBUG [Listener at localhost/46633] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-20 02:56:36,154 DEBUG [Listener at localhost/46633] util.JVMClusterUtil(257): Found active master hash=1252623799, stopped=false 2023-07-20 02:56:36,154 INFO [Listener at localhost/46633] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33365,1689821779383 2023-07-20 02:56:36,156 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 02:56:36,156 INFO [Listener at localhost/46633] procedure2.ProcedureExecutor(629): Stopping 2023-07-20 02:56:36,156 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 02:56:36,156 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:56:36,156 DEBUG [Listener at localhost/46633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0d31ea56 to 127.0.0.1:55129 2023-07-20 02:56:36,156 DEBUG [Listener at localhost/46633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:36,157 INFO [Listener at localhost/46633] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41413,1689821779407' ***** 2023-07-20 02:56:36,157 INFO [Listener at localhost/46633] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 02:56:36,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:56:36,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 02:56:36,162 INFO [RS:0;jenkins-hbase4:41413] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 02:56:36,162 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 02:56:36,162 INFO [RS:0;jenkins-hbase4:41413] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 02:56:36,163 INFO [RS:0;jenkins-hbase4:41413] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 02:56:36,164 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(3305): Received CLOSE for 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:36,164 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(3305): Received CLOSE for 80b6625deaec89ca31a4c1b65e4a6544 2023-07-20 02:56:36,164 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(3305): Received CLOSE for 6908f43230333d97246933b2af61e211 2023-07-20 02:56:36,164 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:36,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 956fe6fd22cc055b7d5c09ea7e2fd4a7, disabling compactions & flushes 2023-07-20 02:56:36,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:36,165 DEBUG [RS:0;jenkins-hbase4:41413] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x412e8259 to 127.0.0.1:55129 2023-07-20 02:56:36,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:36,165 DEBUG [RS:0;jenkins-hbase4:41413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:36,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. after waiting 0 ms 2023-07-20 02:56:36,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:36,165 INFO [RS:0;jenkins-hbase4:41413] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 02:56:36,165 INFO [RS:0;jenkins-hbase4:41413] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 02:56:36,165 INFO [RS:0;jenkins-hbase4:41413] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 02:56:36,165 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-20 02:56:36,173 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-20 02:56:36,173 DEBUG [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1478): Online Regions={956fe6fd22cc055b7d5c09ea7e2fd4a7=hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7., 80b6625deaec89ca31a4c1b65e4a6544=t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544., 1588230740=hbase:meta,,1.1588230740, 6908f43230333d97246933b2af61e211=t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211.} 2023-07-20 02:56:36,174 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 02:56:36,174 DEBUG [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1504): Waiting on 1588230740, 6908f43230333d97246933b2af61e211, 80b6625deaec89ca31a4c1b65e4a6544, 956fe6fd22cc055b7d5c09ea7e2fd4a7 2023-07-20 02:56:36,175 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 02:56:36,175 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 02:56:36,175 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 02:56:36,176 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 02:56:36,176 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-20 02:56:36,182 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/namespace/956fe6fd22cc055b7d5c09ea7e2fd4a7/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-07-20 02:56:36,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:36,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 956fe6fd22cc055b7d5c09ea7e2fd4a7: 2023-07-20 02:56:36,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689821737628.956fe6fd22cc055b7d5c09ea7e2fd4a7. 2023-07-20 02:56:36,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 80b6625deaec89ca31a4c1b65e4a6544, disabling compactions & flushes 2023-07-20 02:56:36,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:36,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:36,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. after waiting 0 ms 2023-07-20 02:56:36,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:36,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 80b6625deaec89ca31a4c1b65e4a6544 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-20 02:56:36,199 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/.tmp/info/0940bca9d6de4b709bc338ecbec4345a 2023-07-20 02:56:36,200 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:56:36,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/.tmp/cf1/da73b39ef81b463dbde4f5e04178293e 2023-07-20 02:56:36,216 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/.tmp/info/0940bca9d6de4b709bc338ecbec4345a as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info/0940bca9d6de4b709bc338ecbec4345a 2023-07-20 02:56:36,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/.tmp/cf1/da73b39ef81b463dbde4f5e04178293e as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1/da73b39ef81b463dbde4f5e04178293e 2023-07-20 02:56:36,222 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/info/0940bca9d6de4b709bc338ecbec4345a, entries=24, sequenceid=43, filesize=7.4 K 2023-07-20 02:56:36,223 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 47ms, sequenceid=43, compaction requested=true 2023-07-20 02:56:36,236 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/cf1/da73b39ef81b463dbde4f5e04178293e, entries=100, sequenceid=21, filesize=8.3 K 2023-07-20 02:56:36,236 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-07-20 02:56:36,237 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 02:56:36,241 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 02:56:36,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 80b6625deaec89ca31a4c1b65e4a6544 in 57ms, sequenceid=21, compaction requested=true 2023-07-20 02:56:36,241 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 02:56:36,242 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-20 02:56:36,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t1_syncup/80b6625deaec89ca31a4c1b65e4a6544/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-07-20 02:56:36,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:36,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 80b6625deaec89ca31a4c1b65e4a6544: 2023-07-20 02:56:36,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689821739674.80b6625deaec89ca31a4c1b65e4a6544. 2023-07-20 02:56:36,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6908f43230333d97246933b2af61e211, disabling compactions & flushes 2023-07-20 02:56:36,253 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:36,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:36,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. after waiting 0 ms 2023-07-20 02:56:36,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:36,253 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6908f43230333d97246933b2af61e211 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-20 02:56:36,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/.tmp/cf1/8267a1651c1b49cfa43acd19635868b9 2023-07-20 02:56:36,277 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/.tmp/cf1/8267a1651c1b49cfa43acd19635868b9 as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1/8267a1651c1b49cfa43acd19635868b9 2023-07-20 02:56:36,282 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/cf1/8267a1651c1b49cfa43acd19635868b9, entries=200, sequenceid=22, filesize=12.2 K 2023-07-20 02:56:36,283 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 6908f43230333d97246933b2af61e211 in 30ms, sequenceid=22, compaction requested=true 2023-07-20 02:56:36,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/data/default/t2_syncup/6908f43230333d97246933b2af61e211/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=18 2023-07-20 02:56:36,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:36,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6908f43230333d97246933b2af61e211: 2023-07-20 02:56:36,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689821740287.6908f43230333d97246933b2af61e211. 2023-07-20 02:56:36,375 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41413,1689821779407; all regions closed. 2023-07-20 02:56:36,381 DEBUG [RS:0;jenkins-hbase4:41413] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs 2023-07-20 02:56:36,381 INFO [RS:0;jenkins-hbase4:41413] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41413%2C1689821779407.meta:.meta(num 1689821779903) 2023-07-20 02:56:36,389 DEBUG [RS:0;jenkins-hbase4:41413] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/oldWALs 2023-07-20 02:56:36,389 INFO [RS:0;jenkins-hbase4:41413] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41413%2C1689821779407:(num 1689821779795) 2023-07-20 02:56:36,389 DEBUG [RS:0;jenkins-hbase4:41413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:36,389 INFO [RS:0;jenkins-hbase4:41413] regionserver.LeaseManager(133): Closed leases 2023-07-20 02:56:36,390 INFO [RS:0;jenkins-hbase4:41413] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 02:56:36,390 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:56:36,390 DEBUG [RS:0;jenkins-hbase4:41413] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4cf1c817 to 127.0.0.1:55129 2023-07-20 02:56:36,390 DEBUG [RS:0;jenkins-hbase4:41413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:36,391 INFO [RS:0;jenkins-hbase4:41413] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41413 2023-07-20 02:56:36,395 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,41413,1689821779407 2023-07-20 02:56:36,395 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca41002a, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:36,395 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca41003d, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:36,395 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:36,395 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10180a7ca410027, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:36,395 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 02:56:36,395 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:56:36,395 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:56:36,395 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 02:56:36,397 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41413,1689821779407] 2023-07-20 02:56:36,397 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41413,1689821779407; numProcessing=1 2023-07-20 02:56:36,398 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,41413,1689821779407 already deleted, retry=false 2023-07-20 02:56:36,398 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41413,1689821779407 expired; onlineServers=0 2023-07-20 02:56:36,398 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33365,1689821779383' ***** 2023-07-20 02:56:36,398 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-20 02:56:36,399 DEBUG [M:0;jenkins-hbase4:33365] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e96083a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 02:56:36,399 INFO [M:0;jenkins-hbase4:33365] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33365,1689821779383 2023-07-20 02:56:36,399 INFO [M:0;jenkins-hbase4:33365] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33365,1689821779383; all regions closed. 2023-07-20 02:56:36,399 DEBUG [M:0;jenkins-hbase4:33365] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 02:56:36,399 DEBUG [M:0;jenkins-hbase4:33365] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-20 02:56:36,399 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-20 02:56:36,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821779578] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689821779578,5,FailOnTimeoutGroup] 2023-07-20 02:56:36,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821779578] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689821779578,5,FailOnTimeoutGroup] 2023-07-20 02:56:36,399 DEBUG [M:0;jenkins-hbase4:33365] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-20 02:56:36,401 INFO [M:0;jenkins-hbase4:33365] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-20 02:56:36,401 INFO [M:0;jenkins-hbase4:33365] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-20 02:56:36,401 INFO [M:0;jenkins-hbase4:33365] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-20 02:56:36,401 DEBUG [M:0;jenkins-hbase4:33365] master.HMaster(1512): Stopping service threads 2023-07-20 02:56:36,401 INFO [M:0;jenkins-hbase4:33365] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-20 02:56:36,401 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-20 02:56:36,402 ERROR [M:0;jenkins-hbase4:33365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-20 02:56:36,402 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 02:56:36,402 INFO [M:0;jenkins-hbase4:33365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-20 02:56:36,402 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-20 02:56:36,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 02:56:36,402 DEBUG [M:0;jenkins-hbase4:33365] zookeeper.ZKUtil(398): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-20 02:56:36,402 WARN [M:0;jenkins-hbase4:33365] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-20 02:56:36,402 INFO [M:0;jenkins-hbase4:33365] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-20 02:56:36,403 INFO [M:0;jenkins-hbase4:33365] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-20 02:56:36,404 DEBUG [M:0;jenkins-hbase4:33365] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 02:56:36,404 INFO [M:0;jenkins-hbase4:33365] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:36,404 DEBUG [M:0;jenkins-hbase4:33365] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:36,404 DEBUG [M:0;jenkins-hbase4:33365] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 02:56:36,404 DEBUG [M:0;jenkins-hbase4:33365] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:36,404 INFO [M:0;jenkins-hbase4:33365] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-07-20 02:56:36,417 INFO [M:0;jenkins-hbase4:33365] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=309 (bloomFilter=true), to=hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/789ea98e43294f05b1c2e58128ea39f6 2023-07-20 02:56:36,422 DEBUG [M:0;jenkins-hbase4:33365] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/789ea98e43294f05b1c2e58128ea39f6 as hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/789ea98e43294f05b1c2e58128ea39f6 2023-07-20 02:56:36,426 INFO [M:0;jenkins-hbase4:33365] regionserver.HStore(1080): Added hdfs://localhost:40517/user/jenkins/test-data/9eca47bb-1b7b-ad74-7085-aa37d145ab16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/789ea98e43294f05b1c2e58128ea39f6, entries=9, sequenceid=309, filesize=5.5 K 2023-07-20 02:56:36,427 INFO [M:0;jenkins-hbase4:33365] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=309, compaction requested=true 2023-07-20 02:56:36,427 INFO [M:0;jenkins-hbase4:33365] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 02:56:36,427 DEBUG [M:0;jenkins-hbase4:33365] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 02:56:36,431 INFO [M:0;jenkins-hbase4:33365] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-20 02:56:36,431 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 02:56:36,431 INFO [M:0;jenkins-hbase4:33365] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33365 2023-07-20 02:56:36,434 DEBUG [M:0;jenkins-hbase4:33365] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,33365,1689821779383 already deleted, retry=false 2023-07-20 02:56:36,594 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:36,594 INFO [M:0;jenkins-hbase4:33365] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33365,1689821779383; zookeeper connection closed. 2023-07-20 02:56:36,594 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): master:33365-0x10180a7ca410033, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:36,694 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:36,694 INFO [RS:0;jenkins-hbase4:41413] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41413,1689821779407; zookeeper connection closed. 2023-07-20 02:56:36,695 DEBUG [Listener at localhost/46633-EventThread] zookeeper.ZKWatcher(600): regionserver:41413-0x10180a7ca410034, quorum=127.0.0.1:55129, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 02:56:36,695 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6fcdce70] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6fcdce70 2023-07-20 02:56:36,695 INFO [Listener at localhost/46633] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-20 02:56:36,695 WARN [Listener at localhost/46633] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 02:56:36,700 INFO [Listener at localhost/46633] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 02:56:36,806 WARN [BP-797302559-172.31.14.131-1689821735277 heartbeating to localhost/127.0.0.1:40517] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 02:56:36,806 WARN [BP-797302559-172.31.14.131-1689821735277 heartbeating to localhost/127.0.0.1:40517] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-797302559-172.31.14.131-1689821735277 (Datanode Uuid d05519fe-bfcd-4db3-937b-29de0dad3891) service to localhost/127.0.0.1:40517 2023-07-20 02:56:36,807 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/cluster_316081d3-021c-b0fe-c72f-4dd54f1e7285/dfs/data/data7/current/BP-797302559-172.31.14.131-1689821735277] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:36,808 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/cluster_316081d3-021c-b0fe-c72f-4dd54f1e7285/dfs/data/data8/current/BP-797302559-172.31.14.131-1689821735277] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:36,809 WARN [Listener at localhost/46633] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 02:56:36,814 INFO [Listener at localhost/46633] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 02:56:36,920 WARN [BP-797302559-172.31.14.131-1689821735277 heartbeating to localhost/127.0.0.1:40517] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 02:56:36,920 WARN [BP-797302559-172.31.14.131-1689821735277 heartbeating to localhost/127.0.0.1:40517] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-797302559-172.31.14.131-1689821735277 (Datanode Uuid 6d82a86a-3163-4833-bb75-a6eb1299bdd9) service to localhost/127.0.0.1:40517 2023-07-20 02:56:36,921 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/cluster_316081d3-021c-b0fe-c72f-4dd54f1e7285/dfs/data/data5/current/BP-797302559-172.31.14.131-1689821735277] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:36,921 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/cluster_316081d3-021c-b0fe-c72f-4dd54f1e7285/dfs/data/data6/current/BP-797302559-172.31.14.131-1689821735277] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:36,922 WARN [Listener at localhost/46633] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 02:56:36,925 INFO [Listener at localhost/46633] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 02:56:37,029 WARN [BP-797302559-172.31.14.131-1689821735277 heartbeating to localhost/127.0.0.1:40517] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 02:56:37,029 WARN [BP-797302559-172.31.14.131-1689821735277 heartbeating to localhost/127.0.0.1:40517] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-797302559-172.31.14.131-1689821735277 (Datanode Uuid f924fba1-da90-407e-a2a9-bca2b317681e) service to localhost/127.0.0.1:40517 2023-07-20 02:56:37,030 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/cluster_316081d3-021c-b0fe-c72f-4dd54f1e7285/dfs/data/data3/current/BP-797302559-172.31.14.131-1689821735277] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:37,030 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/cluster_316081d3-021c-b0fe-c72f-4dd54f1e7285/dfs/data/data4/current/BP-797302559-172.31.14.131-1689821735277] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:37,031 WARN [Listener at localhost/46633] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 02:56:37,034 INFO [Listener at localhost/46633] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 02:56:37,138 WARN [BP-797302559-172.31.14.131-1689821735277 heartbeating to localhost/127.0.0.1:40517] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 02:56:37,139 WARN [BP-797302559-172.31.14.131-1689821735277 heartbeating to localhost/127.0.0.1:40517] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-797302559-172.31.14.131-1689821735277 (Datanode Uuid f7d2b3c4-486a-49ed-81f3-c03d9100d412) service to localhost/127.0.0.1:40517 2023-07-20 02:56:37,139 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/cluster_316081d3-021c-b0fe-c72f-4dd54f1e7285/dfs/data/data1/current/BP-797302559-172.31.14.131-1689821735277] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:37,140 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6e42259d-71de-0527-c306-ef6ee8ea1d42/cluster_316081d3-021c-b0fe-c72f-4dd54f1e7285/dfs/data/data2/current/BP-797302559-172.31.14.131-1689821735277] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:37,173 INFO [Listener at localhost/46633] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 02:56:37,281 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 02:56:37,343 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-20 02:56:37,343 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-20 02:56:37,344 WARN [Listener at localhost/46633] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 02:56:37,346 INFO [Listener at localhost/46633] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 02:56:37,450 WARN [BP-447985409-172.31.14.131-1689821727757 heartbeating to localhost/127.0.0.1:37763] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 02:56:37,450 WARN [BP-447985409-172.31.14.131-1689821727757 heartbeating to localhost/127.0.0.1:37763] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-447985409-172.31.14.131-1689821727757 (Datanode Uuid 6b20068e-fbad-42bc-ad88-732810a3afce) service to localhost/127.0.0.1:37763 2023-07-20 02:56:37,451 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/cluster_5c0383fa-0b5a-b542-a270-52985b3db8f2/dfs/data/data3/current/BP-447985409-172.31.14.131-1689821727757] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:37,451 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/cluster_5c0383fa-0b5a-b542-a270-52985b3db8f2/dfs/data/data4/current/BP-447985409-172.31.14.131-1689821727757] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:37,452 WARN [Listener at localhost/46633] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 02:56:37,454 INFO [Listener at localhost/46633] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 02:56:37,557 WARN [BP-447985409-172.31.14.131-1689821727757 heartbeating to localhost/127.0.0.1:37763] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 02:56:37,557 WARN [BP-447985409-172.31.14.131-1689821727757 heartbeating to localhost/127.0.0.1:37763] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-447985409-172.31.14.131-1689821727757 (Datanode Uuid 3220ed50-8801-4122-923c-5b3408ae42d9) service to localhost/127.0.0.1:37763 2023-07-20 02:56:37,557 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/cluster_5c0383fa-0b5a-b542-a270-52985b3db8f2/dfs/data/data1/current/BP-447985409-172.31.14.131-1689821727757] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:37,558 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ea5dabf-121c-b674-48fc-0c146a0314be/cluster_5c0383fa-0b5a-b542-a270-52985b3db8f2/dfs/data/data2/current/BP-447985409-172.31.14.131-1689821727757] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 02:56:37,568 INFO [Listener at localhost/46633] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 02:56:37,684 INFO [Listener at localhost/46633] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-20 02:56:37,717 INFO [Listener at localhost/46633] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-20 02:56:37,742 INFO [Listener at localhost/46633] hbase.ResourceChecker(175): after: replication.TestReplicationSyncUpTool#testSyncUpTool Thread=186 (was 10) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1211979078) connection to localhost/127.0.0.1:40517 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:40517 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-16 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:40517 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:40517 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x6b831c8f sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1990105035.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x6b831c8f-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x3a61c25d sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1990105035.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:37763 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-17-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x6b831c8f-SendThread(127.0.0.1:55129) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:40517 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x6b43e924 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1990105035.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333-SendThread(127.0.0.1:55129) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1211979078) connection to localhost/127.0.0.1:37763 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:40517 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x4c3a163b-SendThread(127.0.0.1:55129) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:40517 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-17-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x4c3a163b-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-17-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:37763 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:40517 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x4c3a163b sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1990105035.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x3a61c25d-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x3a61c25d-SendThread(127.0.0.1:55129) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1211979078) connection to localhost/127.0.0.1:37763 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:37763 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x0039fd2f sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1990105035.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,39331,1689821732285-SendThread(127.0.0.1:55129) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x6b43e924-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1211979078) connection to localhost/127.0.0.1:40517 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@3a83a04e java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x0039fd2f-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46185,1689821732333-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1211979078) connection to localhost/127.0.0.1:40517 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/46633 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:37763 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41245,1689821771943-SendThread(127.0.0.1:55129) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x6b43e924-SendThread(127.0.0.1:55129) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55129@0x0039fd2f-SendThread(127.0.0.1:55129) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) - Thread LEAK? -, OpenFileDescriptor=659 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 200) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 171) - ProcessCount LEAK? -, AvailableMemoryMB=6321 (was 6947)