2023-07-11 14:35:54,558 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794 2023-07-11 14:35:54,571 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-log-dir so I do NOT create it in target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e 2023-07-11 14:35:54,571 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-log-dir Erasing configuration value by system value. 2023-07-11 14:35:54,571 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-tmp-dir so I do NOT create it in target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e 2023-07-11 14:35:54,571 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-11 14:35:54,572 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e 2023-07-11 14:35:54,590 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-07-11 14:35:54,592 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL timeout: 13 mins 2023-07-11 14:35:54,624 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=172, AvailableMemoryMB=7818 2023-07-11 14:35:54,637 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-log-dir so I do NOT create it in target/test-data/db0654d6-dfa8-c4d3-1a47-740ff033686d 2023-07-11 14:35:54,638 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-log-dir Erasing configuration value by system value. 2023-07-11 14:35:54,638 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-tmp-dir so I do NOT create it in target/test-data/db0654d6-dfa8-c4d3-1a47-740ff033686d 2023-07-11 14:35:54,638 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-11 14:35:54,639 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/db0654d6-dfa8-c4d3-1a47-740ff033686d 2023-07-11 14:35:54,649 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-log-dir so I do NOT create it in target/test-data/1988bd0f-c5a1-7732-dba7-a9cd174b4480 2023-07-11 14:35:54,649 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-log-dir Erasing configuration value by system value. 2023-07-11 14:35:54,649 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-tmp-dir so I do NOT create it in target/test-data/1988bd0f-c5a1-7732-dba7-a9cd174b4480 2023-07-11 14:35:54,650 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-11 14:35:54,650 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1988bd0f-c5a1-7732-dba7-a9cd174b4480 2023-07-11 14:35:54,654 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/cluster_5dfd46ef-8369-c534-0644-37f0dfe3e9ac, deleteOnExit=true 2023-07-11 14:35:54,765 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/cluster_5dfd46ef-8369-c534-0644-37f0dfe3e9ac/zookeeper_0, clientPort=64930, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/cluster_5dfd46ef-8369-c534-0644-37f0dfe3e9ac/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/cluster_5dfd46ef-8369-c534-0644-37f0dfe3e9ac/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-11 14:35:54,780 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=64930 2023-07-11 14:35:54,784 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-11 14:35:54,784 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-11 14:35:54,784 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/test.cache.data in system properties and HBase conf 2023-07-11 14:35:54,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop.tmp.dir in system properties and HBase conf 2023-07-11 14:35:54,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/hadoop.log.dir in system properties and HBase conf 2023-07-11 14:35:54,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-11 14:35:54,786 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-11 14:35:54,786 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-11 14:35:54,886 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-11 14:35:55,264 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-11 14:35:55,267 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-11 14:35:55,268 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-11 14:35:55,268 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-11 14:35:55,268 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-11 14:35:55,268 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-11 14:35:55,269 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-11 14:35:55,269 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-11 14:35:55,269 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-11 14:35:55,269 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-11 14:35:55,270 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/nfs.dump.dir in system properties and HBase conf 2023-07-11 14:35:55,270 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/java.io.tmpdir in system properties and HBase conf 2023-07-11 14:35:55,270 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-11 14:35:55,270 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-11 14:35:55,271 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-11 14:35:55,737 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-11 14:35:55,742 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-11 14:35:56,024 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-11 14:35:56,135 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-11 14:35:56,149 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-11 14:35:56,187 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-11 14:35:56,221 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/java.io.tmpdir/Jetty_localhost_42791_hdfs____.59426d/webapp 2023-07-11 14:35:56,370 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42791 2023-07-11 14:35:56,392 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-11 14:35:56,393 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-11 14:35:56,873 WARN [Listener at localhost/41201] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-11 14:35:56,934 WARN [Listener at localhost/41201] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-11 14:35:56,951 WARN [Listener at localhost/41201] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-11 14:35:56,957 INFO [Listener at localhost/41201] log.Slf4jLog(67): jetty-6.1.26 2023-07-11 14:35:56,961 INFO [Listener at localhost/41201] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/java.io.tmpdir/Jetty_localhost_36703_datanode____t1a74u/webapp 2023-07-11 14:35:57,057 INFO [Listener at localhost/41201] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36703 2023-07-11 14:35:57,357 WARN [Listener at localhost/44623] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-11 14:35:57,367 WARN [Listener at localhost/44623] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-11 14:35:57,369 WARN [Listener at localhost/44623] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-11 14:35:57,371 INFO [Listener at localhost/44623] log.Slf4jLog(67): jetty-6.1.26 2023-07-11 14:35:57,376 INFO [Listener at localhost/44623] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/java.io.tmpdir/Jetty_localhost_46225_datanode____.sn1012/webapp 2023-07-11 14:35:57,472 INFO [Listener at localhost/44623] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46225 2023-07-11 14:35:57,480 WARN [Listener at localhost/36371] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-11 14:35:57,824 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x56f2e2fcb02a591b: Processing first storage report for DS-7fe6473b-be8e-4ffd-9d69-553584c37202 from datanode 5d203a92-b49d-4b88-b811-a29d937209c2 2023-07-11 14:35:57,826 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x56f2e2fcb02a591b: from storage DS-7fe6473b-be8e-4ffd-9d69-553584c37202 node DatanodeRegistration(127.0.0.1:37609, datanodeUuid=5d203a92-b49d-4b88-b811-a29d937209c2, infoPort=38887, infoSecurePort=0, ipcPort=36371, storageInfo=lv=-57;cid=testClusterID;nsid=699190159;c=1689086155813), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-11 14:35:57,826 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x470eab973f5baae3: Processing first storage report for DS-aff66802-3d09-4eb6-949b-2fd2dc490c59 from datanode af9149e4-ca9e-41bc-b3e9-47b65276d12a 2023-07-11 14:35:57,826 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x470eab973f5baae3: from storage DS-aff66802-3d09-4eb6-949b-2fd2dc490c59 node DatanodeRegistration(127.0.0.1:37265, datanodeUuid=af9149e4-ca9e-41bc-b3e9-47b65276d12a, infoPort=36947, infoSecurePort=0, ipcPort=44623, storageInfo=lv=-57;cid=testClusterID;nsid=699190159;c=1689086155813), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-11 14:35:57,826 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x56f2e2fcb02a591b: Processing first storage report for DS-51f1d8b9-fc55-4056-a4ed-ee93091754cd from datanode 5d203a92-b49d-4b88-b811-a29d937209c2 2023-07-11 14:35:57,826 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x56f2e2fcb02a591b: from storage DS-51f1d8b9-fc55-4056-a4ed-ee93091754cd node DatanodeRegistration(127.0.0.1:37609, datanodeUuid=5d203a92-b49d-4b88-b811-a29d937209c2, infoPort=38887, infoSecurePort=0, ipcPort=36371, storageInfo=lv=-57;cid=testClusterID;nsid=699190159;c=1689086155813), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-11 14:35:57,826 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x470eab973f5baae3: Processing first storage report for DS-9fe9cdcb-e5e0-4dd1-9d63-a1465f4c9db3 from datanode af9149e4-ca9e-41bc-b3e9-47b65276d12a 2023-07-11 14:35:57,826 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x470eab973f5baae3: from storage DS-9fe9cdcb-e5e0-4dd1-9d63-a1465f4c9db3 node DatanodeRegistration(127.0.0.1:37265, datanodeUuid=af9149e4-ca9e-41bc-b3e9-47b65276d12a, infoPort=36947, infoSecurePort=0, ipcPort=44623, storageInfo=lv=-57;cid=testClusterID;nsid=699190159;c=1689086155813), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-11 14:35:57,879 DEBUG [Listener at localhost/36371] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794 2023-07-11 14:35:57,891 INFO [Listener at localhost/36371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:35:57,892 INFO [Listener at localhost/36371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:35:58,557 INFO [Listener at localhost/36371] util.FSUtils(471): Created version file at hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d with version=8 2023-07-11 14:35:58,558 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/hbase-staging 2023-07-11 14:35:58,884 INFO [Listener at localhost/36371] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-11 14:35:59,359 INFO [Listener at localhost/36371] client.ConnectionUtils(127): master/jenkins-hbase9:0 server-side Connection retries=45 2023-07-11 14:35:59,390 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:35:59,391 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:35:59,391 INFO [Listener at localhost/36371] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:35:59,391 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:35:59,391 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:35:59,532 INFO [Listener at localhost/36371] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:35:59,608 DEBUG [Listener at localhost/36371] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-11 14:35:59,700 INFO [Listener at localhost/36371] ipc.NettyRpcServer(120): Bind to /172.31.2.10:46131 2023-07-11 14:35:59,710 INFO [Listener at localhost/36371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:35:59,712 INFO [Listener at localhost/36371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:35:59,737 INFO [Listener at localhost/36371] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46131 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:35:59,777 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:461310x0, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:35:59,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:46131-0x10154c21b790000 connected 2023-07-11 14:35:59,802 DEBUG [Listener at localhost/36371] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-11 14:35:59,802 DEBUG [Listener at localhost/36371] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-11 14:35:59,805 DEBUG [Listener at localhost/36371] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-11 14:35:59,813 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46131 2023-07-11 14:35:59,813 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46131 2023-07-11 14:35:59,814 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46131 2023-07-11 14:35:59,814 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46131 2023-07-11 14:35:59,814 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46131 2023-07-11 14:35:59,818 INFO [Listener at localhost/36371] master.HMaster(444): hbase.rootdir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d, hbase.cluster.distributed=false 2023-07-11 14:35:59,883 INFO [Listener at localhost/36371] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=45 2023-07-11 14:35:59,884 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:35:59,884 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:35:59,884 INFO [Listener at localhost/36371] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:35:59,884 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:35:59,884 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:35:59,889 INFO [Listener at localhost/36371] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:35:59,891 INFO [Listener at localhost/36371] ipc.NettyRpcServer(120): Bind to /172.31.2.10:37647 2023-07-11 14:35:59,893 INFO [Listener at localhost/36371] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-11 14:35:59,899 DEBUG [Listener at localhost/36371] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-11 14:35:59,900 INFO [Listener at localhost/36371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:35:59,902 INFO [Listener at localhost/36371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:35:59,904 INFO [Listener at localhost/36371] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37647 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:35:59,907 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:376470x0, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:35:59,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37647-0x10154c21b790001 connected 2023-07-11 14:35:59,908 DEBUG [Listener at localhost/36371] zookeeper.ZKUtil(164): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-11 14:35:59,909 DEBUG [Listener at localhost/36371] zookeeper.ZKUtil(164): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-11 14:35:59,910 DEBUG [Listener at localhost/36371] zookeeper.ZKUtil(164): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-11 14:35:59,910 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37647 2023-07-11 14:35:59,911 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37647 2023-07-11 14:35:59,911 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37647 2023-07-11 14:35:59,912 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37647 2023-07-11 14:35:59,912 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37647 2023-07-11 14:35:59,924 INFO [Listener at localhost/36371] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=45 2023-07-11 14:35:59,924 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:35:59,925 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:35:59,925 INFO [Listener at localhost/36371] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:35:59,925 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:35:59,925 INFO [Listener at localhost/36371] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:35:59,925 INFO [Listener at localhost/36371] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:35:59,927 INFO [Listener at localhost/36371] ipc.NettyRpcServer(120): Bind to /172.31.2.10:42359 2023-07-11 14:35:59,927 INFO [Listener at localhost/36371] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-11 14:35:59,928 DEBUG [Listener at localhost/36371] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-11 14:35:59,929 INFO [Listener at localhost/36371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:35:59,931 INFO [Listener at localhost/36371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:35:59,932 INFO [Listener at localhost/36371] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42359 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:35:59,938 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:423590x0, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:35:59,939 DEBUG [Listener at localhost/36371] zookeeper.ZKUtil(164): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-11 14:35:59,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42359-0x10154c21b790002 connected 2023-07-11 14:35:59,940 DEBUG [Listener at localhost/36371] zookeeper.ZKUtil(164): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-11 14:35:59,941 DEBUG [Listener at localhost/36371] zookeeper.ZKUtil(164): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-11 14:35:59,941 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42359 2023-07-11 14:35:59,942 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42359 2023-07-11 14:35:59,942 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42359 2023-07-11 14:35:59,942 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42359 2023-07-11 14:35:59,942 DEBUG [Listener at localhost/36371] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42359 2023-07-11 14:35:59,944 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:35:59,954 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-11 14:35:59,955 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:35:59,976 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-11 14:35:59,976 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-11 14:35:59,976 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-11 14:35:59,976 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:35:59,977 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-11 14:35:59,979 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase9.apache.org,46131,1689086158709 from backup master directory 2023-07-11 14:35:59,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-11 14:35:59,981 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:35:59,981 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-11 14:35:59,982 WARN [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:35:59,982 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:35:59,985 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-11 14:35:59,986 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-11 14:36:00,084 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/hbase.id with ID: fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 2023-07-11 14:36:00,130 INFO [master/jenkins-hbase9:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:00,147 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:00,189 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x007d23e9 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:00,221 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e822e10, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:00,244 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:00,247 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-11 14:36:00,258 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:00,261 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:00,294 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store-tmp 2023-07-11 14:36:00,324 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:00,325 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-11 14:36:00,325 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:00,325 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:00,325 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-11 14:36:00,325 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:00,325 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:00,325 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:00,327 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:36:00,328 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:00,352 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C46131%2C1689086158709.jenkins-hbase9.apache.org%2C46131%2C1689086158709.regiongroup-0, suffix=, logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,46131,1689086158709, archiveDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/oldWALs, maxLogs=10 2023-07-11 14:36:00,371 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] util.CommonFSUtils$DfsBuilderUtility(753): Could not find replicate method on builder; will not set replicate when creating output stream java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DistributedFileSystem$HdfsDataOutputStreamBuilder.replicate() at java.lang.Class.getMethod(Class.java:1786) at org.apache.hadoop.hbase.util.CommonFSUtils$DfsBuilderUtility.(CommonFSUtils.java:750) at org.apache.hadoop.hbase.util.CommonFSUtils.createForWal(CommonFSUtils.java:802) at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.initOutput(ProtobufLogWriter.java:102) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.FSHLogProvider.createWriter(FSHLogProvider.java:78) at org.apache.hadoop.hbase.regionserver.wal.FSHLog.createWriterInstance(FSHLog.java:307) at org.apache.hadoop.hbase.regionserver.wal.FSHLog.createWriterInstance(FSHLog.java:70) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:195) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:215) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-11 14:36:00,398 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,46131,1689086158709/jenkins-hbase9.apache.org%2C46131%2C1689086158709.jenkins-hbase9.apache.org%2C46131%2C1689086158709.regiongroup-0.1689086160369 2023-07-11 14:36:00,398 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37265,DS-aff66802-3d09-4eb6-949b-2fd2dc490c59,DISK], DatanodeInfoWithStorage[127.0.0.1:37609,DS-7fe6473b-be8e-4ffd-9d69-553584c37202,DISK]] 2023-07-11 14:36:00,399 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:00,399 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:00,403 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:00,404 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:00,459 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:00,466 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-11 14:36:00,489 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-11 14:36:00,501 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:00,507 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:00,509 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:00,525 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:00,530 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-11 14:36:00,531 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60502295, jitterRate=-0.09844554960727692}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-11 14:36:00,531 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:00,532 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-11 14:36:00,551 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-11 14:36:00,551 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-11 14:36:00,554 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-11 14:36:00,556 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-11 14:36:00,589 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 33 msec 2023-07-11 14:36:00,590 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-11 14:36:00,616 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-11 14:36:00,622 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-11 14:36:00,649 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-11 14:36:00,653 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-11 14:36:00,654 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-11 14:36:00,659 INFO [master/jenkins-hbase9:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-11 14:36:00,663 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-11 14:36:00,667 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:00,668 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-11 14:36:00,669 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-11 14:36:00,680 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-11 14:36:00,684 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-11 14:36:00,684 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-11 14:36:00,684 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:00,684 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-11 14:36:00,685 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase9.apache.org,46131,1689086158709, sessionid=0x10154c21b790000, setting cluster-up flag (Was=false) 2023-07-11 14:36:00,699 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:00,709 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-11 14:36:00,710 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:36:00,716 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:00,720 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-11 14:36:00,722 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:36:00,724 WARN [master/jenkins-hbase9:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.hbase-snapshot/.tmp 2023-07-11 14:36:00,746 INFO [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(951): ClusterId : fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 2023-07-11 14:36:00,746 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(951): ClusterId : fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 2023-07-11 14:36:00,750 DEBUG [RS:1;jenkins-hbase9:42359] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-11 14:36:00,750 DEBUG [RS:0;jenkins-hbase9:37647] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-11 14:36:00,761 DEBUG [RS:1;jenkins-hbase9:42359] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-11 14:36:00,761 DEBUG [RS:0;jenkins-hbase9:37647] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-11 14:36:00,761 DEBUG [RS:1;jenkins-hbase9:42359] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-11 14:36:00,761 DEBUG [RS:0;jenkins-hbase9:37647] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-11 14:36:00,765 DEBUG [RS:0;jenkins-hbase9:37647] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-11 14:36:00,765 DEBUG [RS:1;jenkins-hbase9:42359] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-11 14:36:00,766 DEBUG [RS:0;jenkins-hbase9:37647] zookeeper.ReadOnlyZKClient(139): Connect 0x17330197 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:00,766 DEBUG [RS:1;jenkins-hbase9:42359] zookeeper.ReadOnlyZKClient(139): Connect 0x71ca46e5 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:00,773 DEBUG [RS:0;jenkins-hbase9:37647] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@19f4ed01, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:00,773 DEBUG [RS:1;jenkins-hbase9:42359] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7f0db6ac, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:00,773 DEBUG [RS:0;jenkins-hbase9:37647] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1dcb6ae7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:00,774 DEBUG [RS:1;jenkins-hbase9:42359] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7b322d95, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:00,799 DEBUG [RS:1;jenkins-hbase9:42359] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase9:42359 2023-07-11 14:36:00,800 DEBUG [RS:0;jenkins-hbase9:37647] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase9:37647 2023-07-11 14:36:00,802 INFO [RS:1;jenkins-hbase9:42359] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-11 14:36:00,803 INFO [RS:1;jenkins-hbase9:42359] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-11 14:36:00,802 INFO [RS:0;jenkins-hbase9:37647] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-11 14:36:00,803 DEBUG [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1022): About to register with Master. 2023-07-11 14:36:00,803 INFO [RS:0;jenkins-hbase9:37647] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-11 14:36:00,803 DEBUG [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(1022): About to register with Master. 2023-07-11 14:36:00,806 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase9.apache.org,46131,1689086158709 with isa=jenkins-hbase9.apache.org/172.31.2.10:42359, startcode=1689086159924 2023-07-11 14:36:00,806 INFO [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase9.apache.org,46131,1689086158709 with isa=jenkins-hbase9.apache.org/172.31.2.10:37647, startcode=1689086159883 2023-07-11 14:36:00,827 DEBUG [RS:1;jenkins-hbase9:42359] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-11 14:36:00,827 DEBUG [RS:0;jenkins-hbase9:37647] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-11 14:36:00,846 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-11 14:36:00,857 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:00,858 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:00,858 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:00,858 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:00,858 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase9:0, corePoolSize=10, maxPoolSize=10 2023-07-11 14:36:00,858 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:00,859 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:00,859 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:00,863 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689086190863 2023-07-11 14:36:00,866 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-11 14:36:00,870 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-11 14:36:00,871 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-11 14:36:00,879 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:00,879 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-11 14:36:00,886 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-11 14:36:00,887 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-11 14:36:00,887 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-11 14:36:00,887 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-11 14:36:00,889 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:00,891 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-11 14:36:00,892 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-11 14:36:00,892 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-11 14:36:00,897 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-11 14:36:00,898 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-11 14:36:00,900 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086160899,5,FailOnTimeoutGroup] 2023-07-11 14:36:00,900 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086160900,5,FailOnTimeoutGroup] 2023-07-11 14:36:00,900 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:00,902 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-11 14:36:00,903 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:00,904 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:00,926 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-11 14:36:00,927 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-11 14:36:00,928 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d 2023-07-11 14:36:00,946 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:00,949 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-11 14:36:00,952 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/info 2023-07-11 14:36:00,953 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-11 14:36:00,954 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:00,954 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-11 14:36:00,956 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:00,957 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-11 14:36:00,958 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:00,958 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-11 14:36:00,961 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/table 2023-07-11 14:36:00,962 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-11 14:36:00,963 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:00,965 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740 2023-07-11 14:36:00,966 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740 2023-07-11 14:36:00,970 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-11 14:36:00,971 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:41813, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-11 14:36:00,971 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:43847, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-11 14:36:00,972 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-11 14:36:00,976 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-11 14:36:00,977 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65329645, jitterRate=-0.026512429118156433}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-11 14:36:00,977 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-11 14:36:00,978 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-11 14:36:00,978 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-11 14:36:00,978 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-11 14:36:00,978 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-11 14:36:00,979 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-11 14:36:00,981 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-11 14:36:00,982 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-11 14:36:00,983 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46131] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:00,988 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-11 14:36:00,988 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-11 14:36:00,995 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46131] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:01,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-11 14:36:01,001 DEBUG [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d 2023-07-11 14:36:01,001 DEBUG [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d 2023-07-11 14:36:01,001 DEBUG [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41201 2023-07-11 14:36:01,001 DEBUG [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41201 2023-07-11 14:36:01,001 DEBUG [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-11 14:36:01,002 DEBUG [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-11 14:36:01,007 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-11 14:36:01,007 DEBUG [RS:0;jenkins-hbase9:37647] zookeeper.ZKUtil(162): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:01,007 DEBUG [RS:1;jenkins-hbase9:42359] zookeeper.ZKUtil(162): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:01,007 WARN [RS:0;jenkins-hbase9:37647] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:01,008 WARN [RS:1;jenkins-hbase9:42359] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:01,008 INFO [RS:0;jenkins-hbase9:37647] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:01,008 INFO [RS:1;jenkins-hbase9:42359] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:01,008 INFO [RS:0;jenkins-hbase9:37647] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:01,009 INFO [RS:1;jenkins-hbase9:42359] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:01,010 DEBUG [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:01,010 DEBUG [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:01,010 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,37647,1689086159883] 2023-07-11 14:36:01,011 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,42359,1689086159924] 2023-07-11 14:36:01,020 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-11 14:36:01,023 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-11 14:36:01,023 DEBUG [RS:0;jenkins-hbase9:37647] zookeeper.ZKUtil(162): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:01,023 DEBUG [RS:1;jenkins-hbase9:42359] zookeeper.ZKUtil(162): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:01,023 DEBUG [RS:0;jenkins-hbase9:37647] zookeeper.ZKUtil(162): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:01,024 DEBUG [RS:1;jenkins-hbase9:42359] zookeeper.ZKUtil(162): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:01,035 DEBUG [RS:1;jenkins-hbase9:42359] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:01,035 DEBUG [RS:0;jenkins-hbase9:37647] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:01,043 INFO [RS:0;jenkins-hbase9:37647] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-11 14:36:01,043 INFO [RS:1;jenkins-hbase9:42359] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-11 14:36:01,063 INFO [RS:1;jenkins-hbase9:42359] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-11 14:36:01,063 INFO [RS:0;jenkins-hbase9:37647] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-11 14:36:01,067 INFO [RS:0;jenkins-hbase9:37647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-11 14:36:01,067 INFO [RS:1;jenkins-hbase9:42359] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-11 14:36:01,068 INFO [RS:0;jenkins-hbase9:37647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,068 INFO [RS:1;jenkins-hbase9:42359] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,069 INFO [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-11 14:36:01,069 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-11 14:36:01,077 INFO [RS:0;jenkins-hbase9:37647] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,077 INFO [RS:1;jenkins-hbase9:42359] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,077 DEBUG [RS:0;jenkins-hbase9:37647] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,077 DEBUG [RS:1;jenkins-hbase9:42359] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,077 DEBUG [RS:0;jenkins-hbase9:37647] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,077 DEBUG [RS:1;jenkins-hbase9:42359] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:0;jenkins-hbase9:37647] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:1;jenkins-hbase9:42359] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:0;jenkins-hbase9:37647] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:1;jenkins-hbase9:42359] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:0;jenkins-hbase9:37647] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:1;jenkins-hbase9:42359] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:0;jenkins-hbase9:37647] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:01,078 DEBUG [RS:1;jenkins-hbase9:42359] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:01,078 DEBUG [RS:0;jenkins-hbase9:37647] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:1;jenkins-hbase9:42359] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:0;jenkins-hbase9:37647] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:1;jenkins-hbase9:42359] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:0;jenkins-hbase9:37647] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,078 DEBUG [RS:1;jenkins-hbase9:42359] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,079 DEBUG [RS:0;jenkins-hbase9:37647] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,079 DEBUG [RS:1;jenkins-hbase9:42359] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:01,080 INFO [RS:0;jenkins-hbase9:37647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,080 INFO [RS:0;jenkins-hbase9:37647] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,080 INFO [RS:0;jenkins-hbase9:37647] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,081 INFO [RS:1;jenkins-hbase9:42359] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,082 INFO [RS:1;jenkins-hbase9:42359] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,082 INFO [RS:1;jenkins-hbase9:42359] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,098 INFO [RS:0;jenkins-hbase9:37647] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-11 14:36:01,098 INFO [RS:1;jenkins-hbase9:42359] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-11 14:36:01,101 INFO [RS:0;jenkins-hbase9:37647] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,37647,1689086159883-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,101 INFO [RS:1;jenkins-hbase9:42359] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,42359,1689086159924-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,118 INFO [RS:0;jenkins-hbase9:37647] regionserver.Replication(203): jenkins-hbase9.apache.org,37647,1689086159883 started 2023-07-11 14:36:01,118 INFO [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(1637): Serving as jenkins-hbase9.apache.org,37647,1689086159883, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:37647, sessionid=0x10154c21b790001 2023-07-11 14:36:01,119 DEBUG [RS:0;jenkins-hbase9:37647] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-11 14:36:01,119 DEBUG [RS:0;jenkins-hbase9:37647] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:01,119 DEBUG [RS:0;jenkins-hbase9:37647] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,37647,1689086159883' 2023-07-11 14:36:01,119 DEBUG [RS:0;jenkins-hbase9:37647] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-11 14:36:01,120 DEBUG [RS:0;jenkins-hbase9:37647] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-11 14:36:01,120 DEBUG [RS:0;jenkins-hbase9:37647] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-11 14:36:01,120 DEBUG [RS:0;jenkins-hbase9:37647] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-11 14:36:01,120 DEBUG [RS:0;jenkins-hbase9:37647] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:01,120 DEBUG [RS:0;jenkins-hbase9:37647] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,37647,1689086159883' 2023-07-11 14:36:01,120 DEBUG [RS:0;jenkins-hbase9:37647] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-11 14:36:01,121 DEBUG [RS:0;jenkins-hbase9:37647] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-11 14:36:01,121 DEBUG [RS:0;jenkins-hbase9:37647] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-11 14:36:01,122 INFO [RS:0;jenkins-hbase9:37647] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-11 14:36:01,122 INFO [RS:0;jenkins-hbase9:37647] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-11 14:36:01,122 INFO [RS:1;jenkins-hbase9:42359] regionserver.Replication(203): jenkins-hbase9.apache.org,42359,1689086159924 started 2023-07-11 14:36:01,122 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1637): Serving as jenkins-hbase9.apache.org,42359,1689086159924, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:42359, sessionid=0x10154c21b790002 2023-07-11 14:36:01,122 DEBUG [RS:1;jenkins-hbase9:42359] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-11 14:36:01,122 DEBUG [RS:1;jenkins-hbase9:42359] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:01,123 DEBUG [RS:1;jenkins-hbase9:42359] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,42359,1689086159924' 2023-07-11 14:36:01,123 DEBUG [RS:1;jenkins-hbase9:42359] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-11 14:36:01,124 DEBUG [RS:1;jenkins-hbase9:42359] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-11 14:36:01,124 DEBUG [RS:1;jenkins-hbase9:42359] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-11 14:36:01,125 DEBUG [RS:1;jenkins-hbase9:42359] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-11 14:36:01,125 DEBUG [RS:1;jenkins-hbase9:42359] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:01,125 DEBUG [RS:1;jenkins-hbase9:42359] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,42359,1689086159924' 2023-07-11 14:36:01,125 DEBUG [RS:1;jenkins-hbase9:42359] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-11 14:36:01,126 DEBUG [RS:1;jenkins-hbase9:42359] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-11 14:36:01,126 DEBUG [RS:1;jenkins-hbase9:42359] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-11 14:36:01,126 INFO [RS:1;jenkins-hbase9:42359] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-11 14:36:01,127 INFO [RS:1;jenkins-hbase9:42359] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-11 14:36:01,175 DEBUG [jenkins-hbase9:46131] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-07-11 14:36:01,178 DEBUG [jenkins-hbase9:46131] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-07-11 14:36:01,184 DEBUG [jenkins-hbase9:46131] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-11 14:36:01,184 DEBUG [jenkins-hbase9:46131] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-11 14:36:01,184 DEBUG [jenkins-hbase9:46131] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-11 14:36:01,187 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,42359,1689086159924, state=OPENING 2023-07-11 14:36:01,198 DEBUG [PEWorker-2] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-11 14:36:01,200 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:01,201 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-11 14:36:01,204 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,42359,1689086159924}] 2023-07-11 14:36:01,229 INFO [RS:0;jenkins-hbase9:37647] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:01,229 INFO [RS:1;jenkins-hbase9:42359] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:01,232 INFO [RS:0;jenkins-hbase9:37647] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0, suffix=, logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,37647,1689086159883, archiveDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs, maxLogs=10 2023-07-11 14:36:01,232 INFO [RS:1;jenkins-hbase9:42359] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0, suffix=, logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924, archiveDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs, maxLogs=10 2023-07-11 14:36:01,282 INFO [RS:0;jenkins-hbase9:37647] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,37647,1689086159883/jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0.1689086161243 2023-07-11 14:36:01,284 DEBUG [RS:0;jenkins-hbase9:37647] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37609,DS-7fe6473b-be8e-4ffd-9d69-553584c37202,DISK], DatanodeInfoWithStorage[127.0.0.1:37265,DS-aff66802-3d09-4eb6-949b-2fd2dc490c59,DISK]] 2023-07-11 14:36:01,321 INFO [RS:1;jenkins-hbase9:42359] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237 2023-07-11 14:36:01,321 DEBUG [RS:1;jenkins-hbase9:42359] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37609,DS-7fe6473b-be8e-4ffd-9d69-553584c37202,DISK], DatanodeInfoWithStorage[127.0.0.1:37265,DS-aff66802-3d09-4eb6-949b-2fd2dc490c59,DISK]] 2023-07-11 14:36:01,386 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:01,389 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:01,393 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:50522, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-11 14:36:01,404 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-11 14:36:01,404 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:01,404 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:01,404 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:01,408 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C42359%2C1689086159924.meta, suffix=.meta, logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924, archiveDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs, maxLogs=10 2023-07-11 14:36:01,424 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.meta.1689086161410.meta 2023-07-11 14:36:01,424 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37609,DS-7fe6473b-be8e-4ffd-9d69-553584c37202,DISK], DatanodeInfoWithStorage[127.0.0.1:37265,DS-aff66802-3d09-4eb6-949b-2fd2dc490c59,DISK]] 2023-07-11 14:36:01,424 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:01,427 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-11 14:36:01,442 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-11 14:36:01,447 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-11 14:36:01,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-11 14:36:01,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:01,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-11 14:36:01,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-11 14:36:01,455 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-11 14:36:01,456 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/info 2023-07-11 14:36:01,456 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/info 2023-07-11 14:36:01,457 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-11 14:36:01,458 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:01,458 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-11 14:36:01,459 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:01,460 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:01,460 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-11 14:36:01,461 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:01,461 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-11 14:36:01,463 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/table 2023-07-11 14:36:01,463 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/table 2023-07-11 14:36:01,463 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-11 14:36:01,464 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:01,466 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740 2023-07-11 14:36:01,469 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740 2023-07-11 14:36:01,473 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-11 14:36:01,475 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-11 14:36:01,477 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74542562, jitterRate=0.11077073216438293}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-11 14:36:01,477 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-11 14:36:01,488 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689086161379 2023-07-11 14:36:01,505 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-11 14:36:01,506 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-11 14:36:01,507 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,42359,1689086159924, state=OPEN 2023-07-11 14:36:01,510 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-11 14:36:01,510 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-11 14:36:01,515 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-11 14:36:01,515 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,42359,1689086159924 in 306 msec 2023-07-11 14:36:01,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-11 14:36:01,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 515 msec 2023-07-11 14:36:01,526 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 751 msec 2023-07-11 14:36:01,527 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689086161526, completionTime=-1 2023-07-11 14:36:01,527 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-07-11 14:36:01,527 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-11 14:36:01,587 DEBUG [hconnection-0x45d81963-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:01,589 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:50526, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:01,603 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-07-11 14:36:01,604 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689086221604 2023-07-11 14:36:01,604 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689086281604 2023-07-11 14:36:01,604 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 76 msec 2023-07-11 14:36:01,626 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,46131,1689086158709-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,626 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,46131,1689086158709-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,626 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,46131,1689086158709-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,628 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase9:46131, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,628 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:01,633 DEBUG [master/jenkins-hbase9:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-11 14:36:01,641 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-11 14:36:01,643 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:01,653 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-11 14:36:01,655 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-11 14:36:01,657 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-11 14:36:01,680 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:01,682 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72 empty. 2023-07-11 14:36:01,683 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:01,684 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-11 14:36:01,730 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-11 14:36:01,733 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 11796b6d2ba24476446363b9a5701d72, NAME => 'hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp 2023-07-11 14:36:01,756 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:01,756 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 11796b6d2ba24476446363b9a5701d72, disabling compactions & flushes 2023-07-11 14:36:01,756 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:01,757 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:01,757 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. after waiting 0 ms 2023-07-11 14:36:01,757 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:01,757 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:01,757 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 11796b6d2ba24476446363b9a5701d72: 2023-07-11 14:36:01,761 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-11 14:36:01,781 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086161764"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689086161764"}]},"ts":"1689086161764"} 2023-07-11 14:36:01,806 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-11 14:36:01,808 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-11 14:36:01,812 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086161808"}]},"ts":"1689086161808"} 2023-07-11 14:36:01,818 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-11 14:36:01,823 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-07-11 14:36:01,824 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-11 14:36:01,824 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-11 14:36:01,824 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-11 14:36:01,827 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=11796b6d2ba24476446363b9a5701d72, ASSIGN}] 2023-07-11 14:36:01,830 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=11796b6d2ba24476446363b9a5701d72, ASSIGN 2023-07-11 14:36:01,831 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=11796b6d2ba24476446363b9a5701d72, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,42359,1689086159924; forceNewPlan=false, retain=false 2023-07-11 14:36:01,985 INFO [jenkins-hbase9:46131] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-11 14:36:01,985 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=11796b6d2ba24476446363b9a5701d72, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:01,986 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086161985"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086161985"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086161985"}]},"ts":"1689086161985"} 2023-07-11 14:36:01,990 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 11796b6d2ba24476446363b9a5701d72, server=jenkins-hbase9.apache.org,42359,1689086159924}] 2023-07-11 14:36:02,149 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:02,150 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:02,153 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1, suffix=, logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924, archiveDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs, maxLogs=10 2023-07-11 14:36:02,165 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154 2023-07-11 14:36:02,165 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37265,DS-aff66802-3d09-4eb6-949b-2fd2dc490c59,DISK], DatanodeInfoWithStorage[127.0.0.1:37609,DS-7fe6473b-be8e-4ffd-9d69-553584c37202,DISK]] 2023-07-11 14:36:02,165 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 11796b6d2ba24476446363b9a5701d72, NAME => 'hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:02,166 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:02,166 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:02,166 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:02,166 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:02,169 INFO [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:02,171 DEBUG [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/info 2023-07-11 14:36:02,171 DEBUG [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/info 2023-07-11 14:36:02,171 INFO [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11796b6d2ba24476446363b9a5701d72 columnFamilyName info 2023-07-11 14:36:02,172 INFO [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] regionserver.HStore(310): Store=11796b6d2ba24476446363b9a5701d72/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:02,173 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:02,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:02,178 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:02,181 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-11 14:36:02,182 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 11796b6d2ba24476446363b9a5701d72; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73409291, jitterRate=0.09388367831707001}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-11 14:36:02,182 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 11796b6d2ba24476446363b9a5701d72: 2023-07-11 14:36:02,184 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72., pid=6, masterSystemTime=1689086162143 2023-07-11 14:36:02,188 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:02,188 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:02,189 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=11796b6d2ba24476446363b9a5701d72, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:02,190 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086162189"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086162189"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086162189"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086162189"}]},"ts":"1689086162189"} 2023-07-11 14:36:02,200 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-11 14:36:02,200 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 11796b6d2ba24476446363b9a5701d72, server=jenkins-hbase9.apache.org,42359,1689086159924 in 205 msec 2023-07-11 14:36:02,205 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-11 14:36:02,205 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=11796b6d2ba24476446363b9a5701d72, ASSIGN in 374 msec 2023-07-11 14:36:02,207 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-11 14:36:02,207 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086162207"}]},"ts":"1689086162207"} 2023-07-11 14:36:02,210 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-11 14:36:02,215 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-11 14:36:02,220 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 571 msec 2023-07-11 14:36:02,256 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-07-11 14:36:02,257 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-07-11 14:36:02,257 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:02,294 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-11 14:36:02,311 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-11 14:36:02,317 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 31 msec 2023-07-11 14:36:02,329 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-11 14:36:02,340 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-11 14:36:02,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-07-11 14:36:02,355 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-11 14:36:02,358 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-11 14:36:02,358 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.376sec 2023-07-11 14:36:02,361 INFO [master/jenkins-hbase9:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-11 14:36:02,362 INFO [master/jenkins-hbase9:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-11 14:36:02,362 INFO [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-11 14:36:02,363 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,46131,1689086158709-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-11 14:36:02,364 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,46131,1689086158709-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-11 14:36:02,372 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-11 14:36:02,451 DEBUG [Listener at localhost/36371] zookeeper.ReadOnlyZKClient(139): Connect 0x5003310f to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:02,456 DEBUG [Listener at localhost/36371] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@441ec8c2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:02,468 DEBUG [hconnection-0x44ef8184-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:02,479 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:50532, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:02,488 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:36:02,488 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-11 14:36:02,488 INFO [Listener at localhost/36371] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/cluster_dc28c8bb-79eb-6d50-ac76-892a20519f73, deleteOnExit=true 2023-07-11 14:36:02,488 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-11 14:36:02,499 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/test.cache.data in system properties and HBase conf 2023-07-11 14:36:02,499 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/hadoop.tmp.dir in system properties and HBase conf 2023-07-11 14:36:02,500 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/hadoop.log.dir in system properties and HBase conf 2023-07-11 14:36:02,500 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-11 14:36:02,500 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-11 14:36:02,500 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-11 14:36:02,500 DEBUG [Listener at localhost/36371] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-11 14:36:02,500 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-11 14:36:02,501 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-11 14:36:02,501 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-11 14:36:02,501 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-11 14:36:02,501 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-11 14:36:02,501 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-11 14:36:02,501 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-11 14:36:02,501 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-11 14:36:02,501 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-11 14:36:02,501 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/nfs.dump.dir in system properties and HBase conf 2023-07-11 14:36:02,502 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/java.io.tmpdir in system properties and HBase conf 2023-07-11 14:36:02,502 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-11 14:36:02,502 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-11 14:36:02,502 INFO [Listener at localhost/36371] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-11 14:36:02,513 WARN [Listener at localhost/36371] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-11 14:36:02,513 WARN [Listener at localhost/36371] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-11 14:36:02,561 WARN [Listener at localhost/36371] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-11 14:36:02,563 INFO [Listener at localhost/36371] log.Slf4jLog(67): jetty-6.1.26 2023-07-11 14:36:02,568 INFO [Listener at localhost/36371] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/java.io.tmpdir/Jetty_localhost_39783_hdfs____t1fe9i/webapp 2023-07-11 14:36:02,666 INFO [Listener at localhost/36371] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39783 2023-07-11 14:36:02,678 WARN [Listener at localhost/36371] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-11 14:36:02,678 WARN [Listener at localhost/36371] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-11 14:36:02,729 WARN [Listener at localhost/33985] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-11 14:36:02,745 WARN [Listener at localhost/33985] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-11 14:36:02,747 WARN [Listener at localhost/33985] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-11 14:36:02,749 INFO [Listener at localhost/33985] log.Slf4jLog(67): jetty-6.1.26 2023-07-11 14:36:02,754 INFO [Listener at localhost/33985] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/java.io.tmpdir/Jetty_localhost_42851_datanode____6zobjx/webapp 2023-07-11 14:36:02,849 INFO [Listener at localhost/33985] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42851 2023-07-11 14:36:02,859 WARN [Listener at localhost/41001] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-11 14:36:02,976 WARN [Listener at localhost/41001] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-11 14:36:02,982 WARN [Listener at localhost/41001] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-11 14:36:02,983 INFO [Listener at localhost/41001] log.Slf4jLog(67): jetty-6.1.26 2023-07-11 14:36:02,987 INFO [Listener at localhost/41001] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/java.io.tmpdir/Jetty_localhost_38233_datanode____4gtb04/webapp 2023-07-11 14:36:03,084 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6cd0ef33f78af77d: Processing first storage report for DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73 from datanode 2c80c93b-7350-4238-b8c3-7d9d0ea5544f 2023-07-11 14:36:03,084 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6cd0ef33f78af77d: from storage DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73 node DatanodeRegistration(127.0.0.1:45009, datanodeUuid=2c80c93b-7350-4238-b8c3-7d9d0ea5544f, infoPort=38697, infoSecurePort=0, ipcPort=41001, storageInfo=lv=-57;cid=testClusterID;nsid=839656327;c=1689086162520), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-11 14:36:03,085 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6cd0ef33f78af77d: Processing first storage report for DS-4348a8c2-0b6c-4486-a551-a12438b1d95b from datanode 2c80c93b-7350-4238-b8c3-7d9d0ea5544f 2023-07-11 14:36:03,085 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6cd0ef33f78af77d: from storage DS-4348a8c2-0b6c-4486-a551-a12438b1d95b node DatanodeRegistration(127.0.0.1:45009, datanodeUuid=2c80c93b-7350-4238-b8c3-7d9d0ea5544f, infoPort=38697, infoSecurePort=0, ipcPort=41001, storageInfo=lv=-57;cid=testClusterID;nsid=839656327;c=1689086162520), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-11 14:36:03,101 INFO [Listener at localhost/41001] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38233 2023-07-11 14:36:03,110 WARN [Listener at localhost/36459] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-11 14:36:03,144 WARN [Listener at localhost/36459] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-11 14:36:03,148 WARN [Listener at localhost/36459] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-11 14:36:03,151 INFO [Listener at localhost/36459] log.Slf4jLog(67): jetty-6.1.26 2023-07-11 14:36:03,166 INFO [Listener at localhost/36459] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/java.io.tmpdir/Jetty_localhost_36517_datanode____mfjnwf/webapp 2023-07-11 14:36:03,316 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4536711be8662c78: Processing first storage report for DS-e758f896-76b0-41e4-9477-db5ffceae74c from datanode 5295bcd3-d2e9-45a1-ade9-3508dd85f33a 2023-07-11 14:36:03,317 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4536711be8662c78: from storage DS-e758f896-76b0-41e4-9477-db5ffceae74c node DatanodeRegistration(127.0.0.1:34139, datanodeUuid=5295bcd3-d2e9-45a1-ade9-3508dd85f33a, infoPort=36145, infoSecurePort=0, ipcPort=36459, storageInfo=lv=-57;cid=testClusterID;nsid=839656327;c=1689086162520), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-11 14:36:03,317 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4536711be8662c78: Processing first storage report for DS-afd853f1-108d-49da-acb8-d053beba6f15 from datanode 5295bcd3-d2e9-45a1-ade9-3508dd85f33a 2023-07-11 14:36:03,317 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4536711be8662c78: from storage DS-afd853f1-108d-49da-acb8-d053beba6f15 node DatanodeRegistration(127.0.0.1:34139, datanodeUuid=5295bcd3-d2e9-45a1-ade9-3508dd85f33a, infoPort=36145, infoSecurePort=0, ipcPort=36459, storageInfo=lv=-57;cid=testClusterID;nsid=839656327;c=1689086162520), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-11 14:36:03,320 INFO [Listener at localhost/36459] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36517 2023-07-11 14:36:03,333 WARN [Listener at localhost/36819] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-11 14:36:03,394 WARN [Listener at localhost/36819] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-11 14:36:03,397 WARN [Listener at localhost/36819] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-11 14:36:03,399 INFO [Listener at localhost/36819] log.Slf4jLog(67): jetty-6.1.26 2023-07-11 14:36:03,406 INFO [Listener at localhost/36819] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/java.io.tmpdir/Jetty_localhost_45037_datanode____.h3fq5i/webapp 2023-07-11 14:36:03,499 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x51ff08290f02db29: Processing first storage report for DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c from datanode a7aa9ac2-c7cc-4611-a08f-9ba7ee8081a7 2023-07-11 14:36:03,499 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x51ff08290f02db29: from storage DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c node DatanodeRegistration(127.0.0.1:40751, datanodeUuid=a7aa9ac2-c7cc-4611-a08f-9ba7ee8081a7, infoPort=33403, infoSecurePort=0, ipcPort=36819, storageInfo=lv=-57;cid=testClusterID;nsid=839656327;c=1689086162520), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-11 14:36:03,499 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x51ff08290f02db29: Processing first storage report for DS-cab4171d-6d71-474c-b1e2-8ca441ad22c3 from datanode a7aa9ac2-c7cc-4611-a08f-9ba7ee8081a7 2023-07-11 14:36:03,499 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x51ff08290f02db29: from storage DS-cab4171d-6d71-474c-b1e2-8ca441ad22c3 node DatanodeRegistration(127.0.0.1:40751, datanodeUuid=a7aa9ac2-c7cc-4611-a08f-9ba7ee8081a7, infoPort=33403, infoSecurePort=0, ipcPort=36819, storageInfo=lv=-57;cid=testClusterID;nsid=839656327;c=1689086162520), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-11 14:36:03,536 INFO [Listener at localhost/36819] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45037 2023-07-11 14:36:03,546 WARN [Listener at localhost/42861] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-11 14:36:03,674 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x228e49c3bf0ccd88: Processing first storage report for DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda from datanode 3a595fa8-21e9-4073-8979-ed9ec626fc3a 2023-07-11 14:36:03,674 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x228e49c3bf0ccd88: from storage DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda node DatanodeRegistration(127.0.0.1:38067, datanodeUuid=3a595fa8-21e9-4073-8979-ed9ec626fc3a, infoPort=36687, infoSecurePort=0, ipcPort=42861, storageInfo=lv=-57;cid=testClusterID;nsid=839656327;c=1689086162520), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-11 14:36:03,674 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x228e49c3bf0ccd88: Processing first storage report for DS-b053e59e-b102-41a4-ab09-549d8d30b54d from datanode 3a595fa8-21e9-4073-8979-ed9ec626fc3a 2023-07-11 14:36:03,674 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x228e49c3bf0ccd88: from storage DS-b053e59e-b102-41a4-ab09-549d8d30b54d node DatanodeRegistration(127.0.0.1:38067, datanodeUuid=3a595fa8-21e9-4073-8979-ed9ec626fc3a, infoPort=36687, infoSecurePort=0, ipcPort=42861, storageInfo=lv=-57;cid=testClusterID;nsid=839656327;c=1689086162520), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-11 14:36:03,766 DEBUG [Listener at localhost/42861] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e 2023-07-11 14:36:03,767 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,768 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,810 INFO [Listener at localhost/42861] util.FSUtils(471): Created version file at hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902 with version=8 2023-07-11 14:36:03,810 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/hbase-staging 2023-07-11 14:36:03,813 INFO [Listener at localhost/42861] client.ConnectionUtils(127): master/jenkins-hbase9:0 server-side Connection retries=18 2023-07-11 14:36:03,813 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,814 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,814 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:03,814 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,814 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:03,814 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:03,816 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:34835 2023-07-11 14:36:03,816 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,818 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,819 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34835 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:03,823 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:348350x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:03,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:34835-0x10154c21b790007 connected 2023-07-11 14:36:03,838 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:03,839 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:03,840 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-11 14:36:03,841 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34835 2023-07-11 14:36:03,842 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34835 2023-07-11 14:36:03,842 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34835 2023-07-11 14:36:03,842 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34835 2023-07-11 14:36:03,842 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34835 2023-07-11 14:36:03,842 INFO [Listener at localhost/42861] master.HMaster(444): hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902, hbase.cluster.distributed=false 2023-07-11 14:36:03,857 INFO [Listener at localhost/42861] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=18 2023-07-11 14:36:03,857 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,857 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,857 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:03,857 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,857 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:03,857 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:03,859 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:37977 2023-07-11 14:36:03,859 INFO [Listener at localhost/42861] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-11 14:36:03,860 DEBUG [Listener at localhost/42861] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-11 14:36:03,861 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,862 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,863 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37977 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:03,866 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:379770x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:03,867 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:379770x0, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:03,868 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37977-0x10154c21b790008 connected 2023-07-11 14:36:03,868 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:03,869 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-11 14:36:03,869 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37977 2023-07-11 14:36:03,869 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37977 2023-07-11 14:36:03,873 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37977 2023-07-11 14:36:03,874 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37977 2023-07-11 14:36:03,874 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37977 2023-07-11 14:36:03,885 INFO [Listener at localhost/42861] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=18 2023-07-11 14:36:03,886 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,886 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,886 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:03,886 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,886 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:03,886 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:03,888 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:41517 2023-07-11 14:36:03,888 INFO [Listener at localhost/42861] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-11 14:36:03,889 DEBUG [Listener at localhost/42861] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-11 14:36:03,890 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,891 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,892 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41517 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:03,895 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:415170x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:03,897 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41517-0x10154c21b790009 connected 2023-07-11 14:36:03,897 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:03,897 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:03,898 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-11 14:36:03,898 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41517 2023-07-11 14:36:03,898 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41517 2023-07-11 14:36:03,899 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41517 2023-07-11 14:36:03,899 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41517 2023-07-11 14:36:03,899 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41517 2023-07-11 14:36:03,912 INFO [Listener at localhost/42861] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=18 2023-07-11 14:36:03,912 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,912 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,912 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:03,912 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,912 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:03,913 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:03,914 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:34239 2023-07-11 14:36:03,914 INFO [Listener at localhost/42861] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-11 14:36:03,916 DEBUG [Listener at localhost/42861] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-11 14:36:03,916 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,917 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,918 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34239 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:03,922 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:342390x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:03,923 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34239-0x10154c21b79000a connected 2023-07-11 14:36:03,923 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:03,924 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:03,924 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-11 14:36:03,926 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34239 2023-07-11 14:36:03,926 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34239 2023-07-11 14:36:03,926 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34239 2023-07-11 14:36:03,927 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34239 2023-07-11 14:36:03,927 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34239 2023-07-11 14:36:03,940 INFO [Listener at localhost/42861] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=18 2023-07-11 14:36:03,941 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,941 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,941 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:03,941 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:03,941 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:03,941 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:03,943 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:42167 2023-07-11 14:36:03,943 INFO [Listener at localhost/42861] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-11 14:36:03,944 DEBUG [Listener at localhost/42861] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-11 14:36:03,945 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,946 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:03,947 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42167 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:03,952 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:421670x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:03,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42167-0x10154c21b79000b connected 2023-07-11 14:36:03,954 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:03,954 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:03,955 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-11 14:36:03,960 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42167 2023-07-11 14:36:03,960 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42167 2023-07-11 14:36:03,961 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42167 2023-07-11 14:36:03,961 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42167 2023-07-11 14:36:03,961 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42167 2023-07-11 14:36:03,964 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:03,967 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-11 14:36:03,968 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:03,969 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-11 14:36:03,970 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-11 14:36:03,971 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-11 14:36:03,970 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-11 14:36:03,969 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-11 14:36:03,971 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:03,979 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-11 14:36:03,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-11 14:36:03,980 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase9.apache.org,34835,1689086163812 from backup master directory 2023-07-11 14:36:03,982 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:03,982 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-11 14:36:03,982 WARN [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:03,982 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:04,015 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/hbase.id with ID: 6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:04,031 INFO [master/jenkins-hbase9:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:04,034 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:04,054 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x6f08f31c to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:04,058 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c6a08be, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:04,058 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:04,059 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-11 14:36:04,059 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:04,060 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:04,061 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store-tmp 2023-07-11 14:36:04,085 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:04,086 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-11 14:36:04,086 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:04,086 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:04,086 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-11 14:36:04,086 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:04,086 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:04,086 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:04,087 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:04,088 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:04,091 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C34835%2C1689086163812.jenkins-hbase9.apache.org%2C34835%2C1689086163812.regiongroup-0, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,34835,1689086163812, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/oldWALs, maxLogs=10 2023-07-11 14:36:04,109 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,34835,1689086163812/jenkins-hbase9.apache.org%2C34835%2C1689086163812.jenkins-hbase9.apache.org%2C34835%2C1689086163812.regiongroup-0.1689086164091 2023-07-11 14:36:04,109 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK], DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK], DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK]] 2023-07-11 14:36:04,109 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:04,109 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:04,109 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:04,109 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:04,114 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:04,117 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-11 14:36:04,117 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-11 14:36:04,118 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:04,120 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:04,121 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:04,125 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:04,129 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-11 14:36:04,129 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64315028, jitterRate=-0.04163140058517456}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-11 14:36:04,130 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:04,133 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-11 14:36:04,135 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-11 14:36:04,135 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-11 14:36:04,135 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-11 14:36:04,136 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-11 14:36:04,137 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-07-11 14:36:04,137 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-11 14:36:04,138 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-11 14:36:04,139 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-11 14:36:04,150 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-11 14:36:04,151 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-11 14:36:04,151 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-11 14:36:04,151 INFO [master/jenkins-hbase9:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-11 14:36:04,152 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-11 14:36:04,155 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:04,156 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-11 14:36:04,156 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-11 14:36:04,157 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-11 14:36:04,158 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-11 14:36:04,158 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-11 14:36:04,159 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-11 14:36:04,159 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-11 14:36:04,158 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-11 14:36:04,159 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:04,159 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase9.apache.org,34835,1689086163812, sessionid=0x10154c21b790007, setting cluster-up flag (Was=false) 2023-07-11 14:36:04,164 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:04,177 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-11 14:36:04,178 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:04,181 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:04,191 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-11 14:36:04,192 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:04,193 WARN [master/jenkins-hbase9:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.hbase-snapshot/.tmp 2023-07-11 14:36:04,196 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-11 14:36:04,197 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:04,197 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:04,197 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:04,197 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:04,197 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase9:0, corePoolSize=10, maxPoolSize=10 2023-07-11 14:36:04,197 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,197 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:04,197 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,200 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689086194200 2023-07-11 14:36:04,201 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-11 14:36:04,201 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-11 14:36:04,201 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-11 14:36:04,201 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-11 14:36:04,201 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-11 14:36:04,201 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-11 14:36:04,201 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,202 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-11 14:36:04,203 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-11 14:36:04,203 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-11 14:36:04,203 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-11 14:36:04,203 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-11 14:36:04,203 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-11 14:36:04,203 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-11 14:36:04,205 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:04,205 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086164203,5,FailOnTimeoutGroup] 2023-07-11 14:36:04,210 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086164206,5,FailOnTimeoutGroup] 2023-07-11 14:36:04,210 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,210 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-11 14:36:04,210 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,210 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,236 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-11 14:36:04,238 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-11 14:36:04,238 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902 2023-07-11 14:36:04,262 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:04,264 INFO [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(951): ClusterId : 6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:04,265 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-11 14:36:04,266 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(951): ClusterId : 6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:04,271 DEBUG [RS:0;jenkins-hbase9:37977] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-11 14:36:04,271 DEBUG [RS:1;jenkins-hbase9:41517] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-11 14:36:04,272 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info 2023-07-11 14:36:04,273 INFO [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(951): ClusterId : 6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:04,274 DEBUG [RS:3;jenkins-hbase9:42167] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-11 14:36:04,274 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-11 14:36:04,275 INFO [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(951): ClusterId : 6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:04,276 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:04,278 DEBUG [RS:2;jenkins-hbase9:34239] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-11 14:36:04,278 DEBUG [RS:1;jenkins-hbase9:41517] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-11 14:36:04,278 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-11 14:36:04,278 DEBUG [RS:1;jenkins-hbase9:41517] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-11 14:36:04,278 DEBUG [RS:0;jenkins-hbase9:37977] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-11 14:36:04,278 DEBUG [RS:0;jenkins-hbase9:37977] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-11 14:36:04,280 DEBUG [RS:3;jenkins-hbase9:42167] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-11 14:36:04,280 DEBUG [RS:3;jenkins-hbase9:42167] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-11 14:36:04,281 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:04,281 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-11 14:36:04,282 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:04,282 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-11 14:36:04,283 DEBUG [RS:1;jenkins-hbase9:41517] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-11 14:36:04,285 DEBUG [RS:2;jenkins-hbase9:34239] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-11 14:36:04,286 DEBUG [RS:2;jenkins-hbase9:34239] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-11 14:36:04,287 DEBUG [RS:1;jenkins-hbase9:41517] zookeeper.ReadOnlyZKClient(139): Connect 0x48790efc to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:04,287 DEBUG [RS:0;jenkins-hbase9:37977] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-11 14:36:04,288 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table 2023-07-11 14:36:04,288 DEBUG [RS:3;jenkins-hbase9:42167] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-11 14:36:04,289 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-11 14:36:04,290 DEBUG [RS:0;jenkins-hbase9:37977] zookeeper.ReadOnlyZKClient(139): Connect 0x3673105a to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:04,290 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:04,294 DEBUG [RS:2;jenkins-hbase9:34239] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-11 14:36:04,296 DEBUG [RS:3;jenkins-hbase9:42167] zookeeper.ReadOnlyZKClient(139): Connect 0x22ac3689 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:04,296 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740 2023-07-11 14:36:04,296 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740 2023-07-11 14:36:04,300 DEBUG [RS:2;jenkins-hbase9:34239] zookeeper.ReadOnlyZKClient(139): Connect 0x4c0cc103 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:04,302 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-11 14:36:04,307 DEBUG [RS:1;jenkins-hbase9:41517] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@46bee983, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:04,307 DEBUG [RS:1;jenkins-hbase9:41517] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4281658c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:04,310 DEBUG [RS:0;jenkins-hbase9:37977] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@38b4f8e4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:04,311 DEBUG [RS:0;jenkins-hbase9:37977] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7989f2df, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:04,312 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-11 14:36:04,320 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-11 14:36:04,321 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71236506, jitterRate=0.06150665879249573}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-11 14:36:04,321 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-11 14:36:04,321 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-11 14:36:04,321 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-11 14:36:04,321 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-11 14:36:04,321 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-11 14:36:04,321 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-11 14:36:04,323 DEBUG [RS:0;jenkins-hbase9:37977] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase9:37977 2023-07-11 14:36:04,323 INFO [RS:0;jenkins-hbase9:37977] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-11 14:36:04,323 INFO [RS:0;jenkins-hbase9:37977] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-11 14:36:04,323 DEBUG [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(1022): About to register with Master. 2023-07-11 14:36:04,323 DEBUG [RS:1;jenkins-hbase9:41517] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase9:41517 2023-07-11 14:36:04,323 INFO [RS:1;jenkins-hbase9:41517] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-11 14:36:04,323 INFO [RS:1;jenkins-hbase9:41517] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-11 14:36:04,323 DEBUG [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1022): About to register with Master. 2023-07-11 14:36:04,324 DEBUG [RS:3;jenkins-hbase9:42167] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2693a604, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:04,324 INFO [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase9.apache.org,34835,1689086163812 with isa=jenkins-hbase9.apache.org/172.31.2.10:37977, startcode=1689086163856 2023-07-11 14:36:04,324 DEBUG [RS:3;jenkins-hbase9:42167] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@63913cb2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:04,324 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase9.apache.org,34835,1689086163812 with isa=jenkins-hbase9.apache.org/172.31.2.10:41517, startcode=1689086163885 2023-07-11 14:36:04,326 DEBUG [RS:0;jenkins-hbase9:37977] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-11 14:36:04,326 DEBUG [RS:1;jenkins-hbase9:41517] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-11 14:36:04,326 DEBUG [RS:2;jenkins-hbase9:34239] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@42cb26f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:04,327 DEBUG [RS:2;jenkins-hbase9:34239] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@46f4aa73, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:04,335 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-11 14:36:04,335 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-11 14:36:04,336 DEBUG [RS:3;jenkins-hbase9:42167] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase9:42167 2023-07-11 14:36:04,336 INFO [RS:3;jenkins-hbase9:42167] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-11 14:36:04,336 INFO [RS:3;jenkins-hbase9:42167] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-11 14:36:04,336 DEBUG [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1022): About to register with Master. 2023-07-11 14:36:04,337 INFO [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase9.apache.org,34835,1689086163812 with isa=jenkins-hbase9.apache.org/172.31.2.10:42167, startcode=1689086163940 2023-07-11 14:36:04,337 DEBUG [RS:3;jenkins-hbase9:42167] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-11 14:36:04,339 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-11 14:36:04,340 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-11 14:36:04,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-11 14:36:04,345 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53663, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-11 14:36:04,346 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-11 14:36:04,347 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:51009, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-11 14:36:04,349 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34835] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,349 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34835] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,350 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58027, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-11 14:36:04,350 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34835] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,350 DEBUG [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902 2023-07-11 14:36:04,351 DEBUG [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33985 2023-07-11 14:36:04,351 DEBUG [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-11 14:36:04,351 DEBUG [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902 2023-07-11 14:36:04,351 DEBUG [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33985 2023-07-11 14:36:04,351 DEBUG [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-11 14:36:04,351 DEBUG [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902 2023-07-11 14:36:04,351 DEBUG [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33985 2023-07-11 14:36:04,351 DEBUG [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-11 14:36:04,352 DEBUG [RS:2;jenkins-hbase9:34239] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase9:34239 2023-07-11 14:36:04,353 INFO [RS:2;jenkins-hbase9:34239] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-11 14:36:04,353 INFO [RS:2;jenkins-hbase9:34239] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-11 14:36:04,353 DEBUG [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(1022): About to register with Master. 2023-07-11 14:36:04,354 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-11 14:36:04,355 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:04,358 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,37977,1689086163856] 2023-07-11 14:36:04,358 DEBUG [RS:0;jenkins-hbase9:37977] zookeeper.ZKUtil(162): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,358 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,41517,1689086163885] 2023-07-11 14:36:04,358 WARN [RS:0;jenkins-hbase9:37977] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:04,358 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,42167,1689086163940] 2023-07-11 14:36:04,358 INFO [RS:0;jenkins-hbase9:37977] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:04,358 INFO [RS:0;jenkins-hbase9:37977] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:04,358 DEBUG [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,360 INFO [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase9.apache.org,34835,1689086163812 with isa=jenkins-hbase9.apache.org/172.31.2.10:34239, startcode=1689086163911 2023-07-11 14:36:04,360 DEBUG [RS:2;jenkins-hbase9:34239] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-11 14:36:04,361 DEBUG [RS:1;jenkins-hbase9:41517] zookeeper.ZKUtil(162): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,362 DEBUG [RS:3;jenkins-hbase9:42167] zookeeper.ZKUtil(162): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,362 WARN [RS:1;jenkins-hbase9:41517] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:04,362 WARN [RS:3;jenkins-hbase9:42167] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:04,362 INFO [RS:1;jenkins-hbase9:41517] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:04,362 INFO [RS:3;jenkins-hbase9:42167] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:04,362 INFO [RS:1;jenkins-hbase9:41517] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:04,362 INFO [RS:3;jenkins-hbase9:42167] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:04,362 DEBUG [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,362 DEBUG [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,369 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:48449, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-11 14:36:04,371 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34835] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:04,371 DEBUG [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902 2023-07-11 14:36:04,371 DEBUG [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33985 2023-07-11 14:36:04,372 DEBUG [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-11 14:36:04,372 DEBUG [RS:0;jenkins-hbase9:37977] zookeeper.ZKUtil(162): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,372 DEBUG [RS:1;jenkins-hbase9:41517] zookeeper.ZKUtil(162): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,372 DEBUG [RS:0;jenkins-hbase9:37977] zookeeper.ZKUtil(162): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,373 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:04,373 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:04,373 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:04,373 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:04,373 DEBUG [RS:1;jenkins-hbase9:41517] zookeeper.ZKUtil(162): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,373 DEBUG [RS:3;jenkins-hbase9:42167] zookeeper.ZKUtil(162): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,374 DEBUG [RS:0;jenkins-hbase9:37977] zookeeper.ZKUtil(162): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,374 DEBUG [RS:2;jenkins-hbase9:34239] zookeeper.ZKUtil(162): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:04,374 WARN [RS:2;jenkins-hbase9:34239] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:04,374 INFO [RS:2;jenkins-hbase9:34239] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:04,374 DEBUG [RS:1;jenkins-hbase9:41517] zookeeper.ZKUtil(162): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,374 DEBUG [RS:3;jenkins-hbase9:42167] zookeeper.ZKUtil(162): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,374 INFO [RS:2;jenkins-hbase9:34239] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:04,375 DEBUG [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:04,375 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,34239,1689086163911] 2023-07-11 14:36:04,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:04,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:04,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:04,377 DEBUG [RS:1;jenkins-hbase9:41517] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:04,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,377 DEBUG [RS:3;jenkins-hbase9:42167] zookeeper.ZKUtil(162): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,378 DEBUG [RS:0;jenkins-hbase9:37977] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:04,377 INFO [RS:1;jenkins-hbase9:41517] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-11 14:36:04,378 INFO [RS:0;jenkins-hbase9:37977] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-11 14:36:04,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,386 INFO [RS:1;jenkins-hbase9:41517] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-11 14:36:04,386 DEBUG [RS:3;jenkins-hbase9:42167] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:04,387 INFO [RS:3;jenkins-hbase9:42167] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-11 14:36:04,390 INFO [RS:0;jenkins-hbase9:37977] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-11 14:36:04,393 INFO [RS:3;jenkins-hbase9:42167] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-11 14:36:04,393 INFO [RS:1;jenkins-hbase9:41517] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-11 14:36:04,393 INFO [RS:1;jenkins-hbase9:41517] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,399 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-11 14:36:04,399 INFO [RS:3;jenkins-hbase9:42167] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-11 14:36:04,399 INFO [RS:3;jenkins-hbase9:42167] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,400 INFO [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-11 14:36:04,400 INFO [RS:1;jenkins-hbase9:41517] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,401 DEBUG [RS:1;jenkins-hbase9:41517] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,401 DEBUG [RS:1;jenkins-hbase9:41517] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,406 INFO [RS:0;jenkins-hbase9:37977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-11 14:36:04,406 DEBUG [RS:1;jenkins-hbase9:41517] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,406 INFO [RS:0;jenkins-hbase9:37977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,406 DEBUG [RS:1;jenkins-hbase9:41517] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,406 DEBUG [RS:1;jenkins-hbase9:41517] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,406 DEBUG [RS:1;jenkins-hbase9:41517] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:04,406 DEBUG [RS:1;jenkins-hbase9:41517] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,406 DEBUG [RS:1;jenkins-hbase9:41517] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,406 DEBUG [RS:1;jenkins-hbase9:41517] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,406 DEBUG [RS:1;jenkins-hbase9:41517] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,406 DEBUG [RS:2;jenkins-hbase9:34239] zookeeper.ZKUtil(162): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,406 INFO [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-11 14:36:04,407 DEBUG [RS:2;jenkins-hbase9:34239] zookeeper.ZKUtil(162): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,407 INFO [RS:3;jenkins-hbase9:42167] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,408 DEBUG [RS:2;jenkins-hbase9:34239] zookeeper.ZKUtil(162): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:04,408 DEBUG [RS:2;jenkins-hbase9:34239] zookeeper.ZKUtil(162): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,409 DEBUG [RS:2;jenkins-hbase9:34239] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:04,411 INFO [RS:1;jenkins-hbase9:41517] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,411 INFO [RS:1;jenkins-hbase9:41517] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,411 INFO [RS:2;jenkins-hbase9:34239] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-11 14:36:04,411 INFO [RS:1;jenkins-hbase9:41517] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,411 DEBUG [RS:3;jenkins-hbase9:42167] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,411 DEBUG [RS:3;jenkins-hbase9:42167] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,411 DEBUG [RS:3;jenkins-hbase9:42167] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,412 DEBUG [RS:3;jenkins-hbase9:42167] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,412 DEBUG [RS:3;jenkins-hbase9:42167] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,412 DEBUG [RS:3;jenkins-hbase9:42167] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:04,412 DEBUG [RS:3;jenkins-hbase9:42167] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,412 DEBUG [RS:3;jenkins-hbase9:42167] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,412 DEBUG [RS:3;jenkins-hbase9:42167] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,412 DEBUG [RS:3;jenkins-hbase9:42167] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,422 INFO [RS:0;jenkins-hbase9:37977] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,422 INFO [RS:2;jenkins-hbase9:34239] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-11 14:36:04,422 DEBUG [RS:0;jenkins-hbase9:37977] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,422 DEBUG [RS:0;jenkins-hbase9:37977] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,423 DEBUG [RS:0;jenkins-hbase9:37977] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,423 DEBUG [RS:0;jenkins-hbase9:37977] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,423 DEBUG [RS:0;jenkins-hbase9:37977] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,423 DEBUG [RS:0;jenkins-hbase9:37977] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:04,423 DEBUG [RS:0;jenkins-hbase9:37977] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,423 DEBUG [RS:0;jenkins-hbase9:37977] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,423 DEBUG [RS:0;jenkins-hbase9:37977] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,423 DEBUG [RS:0;jenkins-hbase9:37977] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,423 INFO [RS:2;jenkins-hbase9:34239] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-11 14:36:04,423 INFO [RS:2;jenkins-hbase9:34239] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,425 INFO [RS:3;jenkins-hbase9:42167] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,426 INFO [RS:3;jenkins-hbase9:42167] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,426 INFO [RS:3;jenkins-hbase9:42167] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,426 INFO [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-11 14:36:04,430 INFO [RS:0;jenkins-hbase9:37977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,430 INFO [RS:0;jenkins-hbase9:37977] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,431 INFO [RS:0;jenkins-hbase9:37977] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,431 INFO [RS:2;jenkins-hbase9:34239] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,431 DEBUG [RS:2;jenkins-hbase9:34239] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,431 DEBUG [RS:2;jenkins-hbase9:34239] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,431 DEBUG [RS:2;jenkins-hbase9:34239] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,431 DEBUG [RS:2;jenkins-hbase9:34239] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,432 DEBUG [RS:2;jenkins-hbase9:34239] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,432 DEBUG [RS:2;jenkins-hbase9:34239] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:04,432 DEBUG [RS:2;jenkins-hbase9:34239] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,432 DEBUG [RS:2;jenkins-hbase9:34239] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,432 DEBUG [RS:2;jenkins-hbase9:34239] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,432 DEBUG [RS:2;jenkins-hbase9:34239] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:04,437 INFO [RS:2;jenkins-hbase9:34239] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,437 INFO [RS:2;jenkins-hbase9:34239] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,437 INFO [RS:2;jenkins-hbase9:34239] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,438 INFO [RS:1;jenkins-hbase9:41517] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-11 14:36:04,438 INFO [RS:1;jenkins-hbase9:41517] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,41517,1689086163885-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,448 INFO [RS:3;jenkins-hbase9:42167] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-11 14:36:04,448 INFO [RS:3;jenkins-hbase9:42167] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,42167,1689086163940-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,450 INFO [RS:0;jenkins-hbase9:37977] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-11 14:36:04,450 INFO [RS:0;jenkins-hbase9:37977] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,37977,1689086163856-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,452 INFO [RS:2;jenkins-hbase9:34239] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-11 14:36:04,452 INFO [RS:2;jenkins-hbase9:34239] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,34239,1689086163911-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,454 INFO [RS:1;jenkins-hbase9:41517] regionserver.Replication(203): jenkins-hbase9.apache.org,41517,1689086163885 started 2023-07-11 14:36:04,454 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1637): Serving as jenkins-hbase9.apache.org,41517,1689086163885, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:41517, sessionid=0x10154c21b790009 2023-07-11 14:36:04,454 DEBUG [RS:1;jenkins-hbase9:41517] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-11 14:36:04,455 DEBUG [RS:1;jenkins-hbase9:41517] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,455 DEBUG [RS:1;jenkins-hbase9:41517] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,41517,1689086163885' 2023-07-11 14:36:04,455 DEBUG [RS:1;jenkins-hbase9:41517] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-11 14:36:04,455 DEBUG [RS:1;jenkins-hbase9:41517] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-11 14:36:04,457 DEBUG [RS:1;jenkins-hbase9:41517] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-11 14:36:04,458 DEBUG [RS:1;jenkins-hbase9:41517] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-11 14:36:04,458 DEBUG [RS:1;jenkins-hbase9:41517] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,458 DEBUG [RS:1;jenkins-hbase9:41517] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,41517,1689086163885' 2023-07-11 14:36:04,458 DEBUG [RS:1;jenkins-hbase9:41517] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-11 14:36:04,459 DEBUG [RS:1;jenkins-hbase9:41517] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-11 14:36:04,462 DEBUG [RS:1;jenkins-hbase9:41517] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-11 14:36:04,462 INFO [RS:1;jenkins-hbase9:41517] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-11 14:36:04,462 INFO [RS:1;jenkins-hbase9:41517] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-11 14:36:04,469 INFO [RS:3;jenkins-hbase9:42167] regionserver.Replication(203): jenkins-hbase9.apache.org,42167,1689086163940 started 2023-07-11 14:36:04,470 INFO [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1637): Serving as jenkins-hbase9.apache.org,42167,1689086163940, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:42167, sessionid=0x10154c21b79000b 2023-07-11 14:36:04,471 INFO [RS:0;jenkins-hbase9:37977] regionserver.Replication(203): jenkins-hbase9.apache.org,37977,1689086163856 started 2023-07-11 14:36:04,471 INFO [RS:2;jenkins-hbase9:34239] regionserver.Replication(203): jenkins-hbase9.apache.org,34239,1689086163911 started 2023-07-11 14:36:04,471 DEBUG [RS:3;jenkins-hbase9:42167] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-11 14:36:04,471 INFO [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(1637): Serving as jenkins-hbase9.apache.org,34239,1689086163911, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:34239, sessionid=0x10154c21b79000a 2023-07-11 14:36:04,471 INFO [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(1637): Serving as jenkins-hbase9.apache.org,37977,1689086163856, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:37977, sessionid=0x10154c21b790008 2023-07-11 14:36:04,472 DEBUG [RS:2;jenkins-hbase9:34239] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-11 14:36:04,472 DEBUG [RS:3;jenkins-hbase9:42167] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,472 DEBUG [RS:0;jenkins-hbase9:37977] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-11 14:36:04,472 DEBUG [RS:0;jenkins-hbase9:37977] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,472 DEBUG [RS:2;jenkins-hbase9:34239] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:04,472 DEBUG [RS:0;jenkins-hbase9:37977] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,37977,1689086163856' 2023-07-11 14:36:04,472 DEBUG [RS:3;jenkins-hbase9:42167] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,42167,1689086163940' 2023-07-11 14:36:04,472 DEBUG [RS:3;jenkins-hbase9:42167] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-11 14:36:04,472 DEBUG [RS:0;jenkins-hbase9:37977] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-11 14:36:04,472 DEBUG [RS:2;jenkins-hbase9:34239] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,34239,1689086163911' 2023-07-11 14:36:04,472 DEBUG [RS:2;jenkins-hbase9:34239] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-11 14:36:04,472 DEBUG [RS:3;jenkins-hbase9:42167] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-11 14:36:04,473 DEBUG [RS:0;jenkins-hbase9:37977] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-11 14:36:04,473 DEBUG [RS:2;jenkins-hbase9:34239] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-11 14:36:04,473 DEBUG [RS:3;jenkins-hbase9:42167] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-11 14:36:04,473 DEBUG [RS:3;jenkins-hbase9:42167] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-11 14:36:04,473 DEBUG [RS:3;jenkins-hbase9:42167] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:04,473 DEBUG [RS:3;jenkins-hbase9:42167] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,42167,1689086163940' 2023-07-11 14:36:04,473 DEBUG [RS:0;jenkins-hbase9:37977] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-11 14:36:04,473 DEBUG [RS:0;jenkins-hbase9:37977] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-11 14:36:04,473 DEBUG [RS:2;jenkins-hbase9:34239] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-11 14:36:04,473 DEBUG [RS:2;jenkins-hbase9:34239] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-11 14:36:04,473 DEBUG [RS:2;jenkins-hbase9:34239] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:04,473 DEBUG [RS:2;jenkins-hbase9:34239] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,34239,1689086163911' 2023-07-11 14:36:04,473 DEBUG [RS:2;jenkins-hbase9:34239] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-11 14:36:04,473 DEBUG [RS:3;jenkins-hbase9:42167] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-11 14:36:04,473 DEBUG [RS:0;jenkins-hbase9:37977] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:04,474 DEBUG [RS:0;jenkins-hbase9:37977] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,37977,1689086163856' 2023-07-11 14:36:04,474 DEBUG [RS:0;jenkins-hbase9:37977] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-11 14:36:04,474 DEBUG [RS:2;jenkins-hbase9:34239] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-11 14:36:04,474 DEBUG [RS:3;jenkins-hbase9:42167] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-11 14:36:04,474 DEBUG [RS:0;jenkins-hbase9:37977] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-11 14:36:04,474 DEBUG [RS:2;jenkins-hbase9:34239] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-11 14:36:04,474 INFO [RS:2;jenkins-hbase9:34239] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-11 14:36:04,474 INFO [RS:2;jenkins-hbase9:34239] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-11 14:36:04,474 DEBUG [RS:3;jenkins-hbase9:42167] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-11 14:36:04,474 INFO [RS:3;jenkins-hbase9:42167] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-11 14:36:04,474 INFO [RS:3;jenkins-hbase9:42167] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-11 14:36:04,475 DEBUG [RS:0;jenkins-hbase9:37977] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-11 14:36:04,475 INFO [RS:0;jenkins-hbase9:37977] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-11 14:36:04,475 INFO [RS:0;jenkins-hbase9:37977] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-11 14:36:04,504 DEBUG [jenkins-hbase9:34835] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-07-11 14:36:04,504 DEBUG [jenkins-hbase9:34835] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-07-11 14:36:04,506 DEBUG [jenkins-hbase9:34835] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-11 14:36:04,506 DEBUG [jenkins-hbase9:34835] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-11 14:36:04,507 DEBUG [jenkins-hbase9:34835] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-11 14:36:04,507 DEBUG [jenkins-hbase9:34835] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-11 14:36:04,507 DEBUG [jenkins-hbase9:34835] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-11 14:36:04,508 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,41517,1689086163885, state=OPENING 2023-07-11 14:36:04,509 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-11 14:36:04,511 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:04,511 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-11 14:36:04,511 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,41517,1689086163885}] 2023-07-11 14:36:04,562 INFO [RS:1;jenkins-hbase9:41517] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:04,565 INFO [RS:1;jenkins-hbase9:41517] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C41517%2C1689086163885.jenkins-hbase9.apache.org%2C41517%2C1689086163885.regiongroup-0, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:04,574 INFO [RS:2;jenkins-hbase9:34239] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:04,575 INFO [RS:3;jenkins-hbase9:42167] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:04,575 INFO [RS:0;jenkins-hbase9:37977] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:04,582 INFO [RS:3;jenkins-hbase9:42167] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C42167%2C1689086163940.jenkins-hbase9.apache.org%2C42167%2C1689086163940.regiongroup-0, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:04,582 INFO [RS:0;jenkins-hbase9:37977] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C37977%2C1689086163856.jenkins-hbase9.apache.org%2C37977%2C1689086163856.regiongroup-0, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,37977,1689086163856, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:04,582 INFO [RS:2;jenkins-hbase9:34239] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C34239%2C1689086163911.jenkins-hbase9.apache.org%2C34239%2C1689086163911.regiongroup-0, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,34239,1689086163911, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:04,583 INFO [RS:1;jenkins-hbase9:41517] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885/jenkins-hbase9.apache.org%2C41517%2C1689086163885.jenkins-hbase9.apache.org%2C41517%2C1689086163885.regiongroup-0.1689086164566 2023-07-11 14:36:04,584 DEBUG [RS:1;jenkins-hbase9:41517] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK], DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK], DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK]] 2023-07-11 14:36:04,651 INFO [RS:3;jenkins-hbase9:42167] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940/jenkins-hbase9.apache.org%2C42167%2C1689086163940.jenkins-hbase9.apache.org%2C42167%2C1689086163940.regiongroup-0.1689086164588 2023-07-11 14:36:04,652 INFO [RS:2;jenkins-hbase9:34239] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,34239,1689086163911/jenkins-hbase9.apache.org%2C34239%2C1689086163911.jenkins-hbase9.apache.org%2C34239%2C1689086163911.regiongroup-0.1689086164589 2023-07-11 14:36:04,652 DEBUG [RS:3;jenkins-hbase9:42167] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK], DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK], DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK]] 2023-07-11 14:36:04,652 DEBUG [RS:2;jenkins-hbase9:34239] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK], DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK], DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK]] 2023-07-11 14:36:04,653 INFO [RS:0;jenkins-hbase9:37977] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,37977,1689086163856/jenkins-hbase9.apache.org%2C37977%2C1689086163856.jenkins-hbase9.apache.org%2C37977%2C1689086163856.regiongroup-0.1689086164589 2023-07-11 14:36:04,653 DEBUG [RS:0;jenkins-hbase9:37977] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK], DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK], DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK]] 2023-07-11 14:36:04,668 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,668 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:04,671 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55836, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-11 14:36:04,676 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-11 14:36:04,676 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:04,676 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:04,676 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:04,678 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C41517%2C1689086163885.meta, suffix=.meta, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:04,690 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885/jenkins-hbase9.apache.org%2C41517%2C1689086163885.meta.1689086164679.meta 2023-07-11 14:36:04,690 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK], DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK], DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK]] 2023-07-11 14:36:04,691 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:04,691 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-11 14:36:04,691 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-11 14:36:04,694 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-11 14:36:04,694 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-11 14:36:04,694 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:04,694 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-11 14:36:04,694 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-11 14:36:04,697 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-11 14:36:04,698 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info 2023-07-11 14:36:04,698 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info 2023-07-11 14:36:04,699 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-11 14:36:04,700 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:04,700 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-11 14:36:04,701 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:04,701 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:04,702 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-11 14:36:04,703 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:04,703 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-11 14:36:04,704 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table 2023-07-11 14:36:04,704 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table 2023-07-11 14:36:04,704 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-11 14:36:04,705 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:04,707 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740 2023-07-11 14:36:04,709 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740 2023-07-11 14:36:04,712 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-11 14:36:04,714 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-11 14:36:04,715 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68982195, jitterRate=0.027914807200431824}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-11 14:36:04,715 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-11 14:36:04,718 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689086164668 2023-07-11 14:36:04,724 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-11 14:36:04,725 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-11 14:36:04,726 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,41517,1689086163885, state=OPEN 2023-07-11 14:36:04,733 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-11 14:36:04,733 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-11 14:36:04,737 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-11 14:36:04,737 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,41517,1689086163885 in 222 msec 2023-07-11 14:36:04,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-11 14:36:04,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 397 msec 2023-07-11 14:36:04,745 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 549 msec 2023-07-11 14:36:04,745 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689086164745, completionTime=-1 2023-07-11 14:36:04,745 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-07-11 14:36:04,746 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-11 14:36:04,750 DEBUG [hconnection-0x2f8a4eb9-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:04,753 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55850, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:04,755 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-07-11 14:36:04,755 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689086224755 2023-07-11 14:36:04,755 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689086284755 2023-07-11 14:36:04,755 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 9 msec 2023-07-11 14:36:04,763 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,34835,1689086163812-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,763 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,34835,1689086163812-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,763 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,34835,1689086163812-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,763 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase9:34835, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,763 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:04,763 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-11 14:36:04,763 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:04,764 DEBUG [master/jenkins-hbase9:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-11 14:36:04,765 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-11 14:36:04,767 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-11 14:36:04,768 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-11 14:36:04,770 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:04,771 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7 empty. 2023-07-11 14:36:04,772 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:04,772 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-11 14:36:04,790 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-11 14:36:04,791 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => d4185bdf44467f073faaa5de49a197d7, NAME => 'hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp 2023-07-11 14:36:04,806 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:04,806 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing d4185bdf44467f073faaa5de49a197d7, disabling compactions & flushes 2023-07-11 14:36:04,806 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:04,806 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:04,806 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. after waiting 0 ms 2023-07-11 14:36:04,806 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:04,806 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:04,806 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for d4185bdf44467f073faaa5de49a197d7: 2023-07-11 14:36:04,810 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-11 14:36:04,812 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086164812"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689086164812"}]},"ts":"1689086164812"} 2023-07-11 14:36:04,816 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-11 14:36:04,818 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-11 14:36:04,818 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086164818"}]},"ts":"1689086164818"} 2023-07-11 14:36:04,821 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-11 14:36:04,825 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-07-11 14:36:04,826 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-11 14:36:04,826 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-11 14:36:04,826 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-11 14:36:04,826 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-11 14:36:04,826 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-11 14:36:04,827 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN}] 2023-07-11 14:36:04,830 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN 2023-07-11 14:36:04,831 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41517,1689086163885; forceNewPlan=false, retain=false 2023-07-11 14:36:04,981 INFO [jenkins-hbase9:34835] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-11 14:36:04,983 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d4185bdf44467f073faaa5de49a197d7, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:04,984 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086164983"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086164983"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086164983"}]},"ts":"1689086164983"} 2023-07-11 14:36:04,987 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure d4185bdf44467f073faaa5de49a197d7, server=jenkins-hbase9.apache.org,41517,1689086163885}] 2023-07-11 14:36:05,148 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:05,148 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:05,150 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C41517%2C1689086163885.jenkins-hbase9.apache.org%2C41517%2C1689086163885.regiongroup-1, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:05,164 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885/jenkins-hbase9.apache.org%2C41517%2C1689086163885.jenkins-hbase9.apache.org%2C41517%2C1689086163885.regiongroup-1.1689086165150 2023-07-11 14:36:05,164 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK], DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK], DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK]] 2023-07-11 14:36:05,165 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d4185bdf44467f073faaa5de49a197d7, NAME => 'hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:05,165 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:05,165 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:05,165 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:05,165 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:05,167 INFO [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:05,169 DEBUG [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/info 2023-07-11 14:36:05,169 DEBUG [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/info 2023-07-11 14:36:05,170 INFO [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4185bdf44467f073faaa5de49a197d7 columnFamilyName info 2023-07-11 14:36:05,171 INFO [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] regionserver.HStore(310): Store=d4185bdf44467f073faaa5de49a197d7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:05,173 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:05,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:05,178 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:05,180 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-11 14:36:05,181 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened d4185bdf44467f073faaa5de49a197d7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66505414, jitterRate=-0.008992105722427368}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-11 14:36:05,181 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for d4185bdf44467f073faaa5de49a197d7: 2023-07-11 14:36:05,184 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7., pid=6, masterSystemTime=1689086165141 2023-07-11 14:36:05,187 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:05,187 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:05,189 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d4185bdf44467f073faaa5de49a197d7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:05,189 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086165188"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086165188"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086165188"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086165188"}]},"ts":"1689086165188"} 2023-07-11 14:36:05,196 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-11 14:36:05,196 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure d4185bdf44467f073faaa5de49a197d7, server=jenkins-hbase9.apache.org,41517,1689086163885 in 205 msec 2023-07-11 14:36:05,200 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-11 14:36:05,202 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN in 369 msec 2023-07-11 14:36:05,203 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-11 14:36:05,203 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086165203"}]},"ts":"1689086165203"} 2023-07-11 14:36:05,205 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-11 14:36:05,209 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-11 14:36:05,211 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 446 msec 2023-07-11 14:36:05,266 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-07-11 14:36:05,271 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-07-11 14:36:05,271 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:05,278 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-11 14:36:05,293 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-11 14:36:05,299 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 20 msec 2023-07-11 14:36:05,311 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-11 14:36:05,323 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-11 14:36:05,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-07-11 14:36:05,335 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-11 14:36:05,338 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-11 14:36:05,338 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.356sec 2023-07-11 14:36:05,339 INFO [master/jenkins-hbase9:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-11 14:36:05,339 INFO [master/jenkins-hbase9:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-11 14:36:05,339 INFO [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-11 14:36:05,339 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,34835,1689086163812-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-11 14:36:05,340 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,34835,1689086163812-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-11 14:36:05,343 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-11 14:36:05,373 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(139): Connect 0x1ac7aba1 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:05,379 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@22eee801, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:05,382 DEBUG [hconnection-0x1e6db056-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:05,385 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55854, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:05,387 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:05,398 DEBUG [Listener at localhost/42861] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-11 14:36:05,401 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:41492, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-11 14:36:05,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:05,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-11 14:36:05,421 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-11 14:36:05,423 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-11 14:36:05,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(700): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-11 14:36:05,426 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:05,427 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73 empty. 2023-07-11 14:36:05,427 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:05,427 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-11 14:36:05,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-11 14:36:05,453 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-11 14:36:05,454 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3bfd16063a0672430ddd816d03e73f73, NAME => 't1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp 2023-07-11 14:36:05,476 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:05,476 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 3bfd16063a0672430ddd816d03e73f73, disabling compactions & flushes 2023-07-11 14:36:05,476 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:05,476 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:05,476 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. after waiting 0 ms 2023-07-11 14:36:05,476 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:05,476 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:05,476 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 3bfd16063a0672430ddd816d03e73f73: 2023-07-11 14:36:05,480 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-11 14:36:05,482 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086165482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689086165482"}]},"ts":"1689086165482"} 2023-07-11 14:36:05,485 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-11 14:36:05,486 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-11 14:36:05,487 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086165486"}]},"ts":"1689086165486"} 2023-07-11 14:36:05,489 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-11 14:36:05,494 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-07-11 14:36:05,495 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-11 14:36:05,495 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-11 14:36:05,496 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-11 14:36:05,496 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=3bfd16063a0672430ddd816d03e73f73, ASSIGN}] 2023-07-11 14:36:05,498 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=3bfd16063a0672430ddd816d03e73f73, ASSIGN 2023-07-11 14:36:05,499 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=3bfd16063a0672430ddd816d03e73f73, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,42359,1689086159924; forceNewPlan=false, retain=false 2023-07-11 14:36:05,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-11 14:36:05,649 INFO [jenkins-hbase9:46131] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-11 14:36:05,652 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=3bfd16063a0672430ddd816d03e73f73, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:05,652 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086165651"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086165651"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086165651"}]},"ts":"1689086165651"} 2023-07-11 14:36:05,655 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 3bfd16063a0672430ddd816d03e73f73, server=jenkins-hbase9.apache.org,42359,1689086159924}] 2023-07-11 14:36:05,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-11 14:36:05,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:05,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3bfd16063a0672430ddd816d03e73f73, NAME => 't1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:05,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:05,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:05,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:05,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:05,819 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:05,821 DEBUG [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/cf1 2023-07-11 14:36:05,821 DEBUG [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/cf1 2023-07-11 14:36:05,822 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bfd16063a0672430ddd816d03e73f73 columnFamilyName cf1 2023-07-11 14:36:05,822 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] regionserver.HStore(310): Store=3bfd16063a0672430ddd816d03e73f73/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:05,822 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:05,824 DEBUG [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/norep 2023-07-11 14:36:05,824 DEBUG [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/norep 2023-07-11 14:36:05,825 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bfd16063a0672430ddd816d03e73f73 columnFamilyName norep 2023-07-11 14:36:05,825 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] regionserver.HStore(310): Store=3bfd16063a0672430ddd816d03e73f73/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:05,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:05,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:05,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-11 14:36:05,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:05,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-11 14:36:05,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 3bfd16063a0672430ddd816d03e73f73; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65963124, jitterRate=-0.017072856426239014}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-11 14:36:05,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 3bfd16063a0672430ddd816d03e73f73: 2023-07-11 14:36:05,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73., pid=11, masterSystemTime=1689086165808 2023-07-11 14:36:05,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:05,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:05,845 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=3bfd16063a0672430ddd816d03e73f73, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:05,845 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086165845"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086165845"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086165845"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086165845"}]},"ts":"1689086165845"} 2023-07-11 14:36:05,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-11 14:36:05,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 3bfd16063a0672430ddd816d03e73f73, server=jenkins-hbase9.apache.org,42359,1689086159924 in 193 msec 2023-07-11 14:36:05,855 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-11 14:36:05,855 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=3bfd16063a0672430ddd816d03e73f73, ASSIGN in 355 msec 2023-07-11 14:36:05,856 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-11 14:36:05,857 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086165857"}]},"ts":"1689086165857"} 2023-07-11 14:36:05,859 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-11 14:36:05,862 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-11 14:36:05,864 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 446 msec 2023-07-11 14:36:06,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-11 14:36:06,050 INFO [Listener at localhost/42861] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-11 14:36:06,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:06,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-11 14:36:06,057 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-11 14:36:06,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(700): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-11 14:36:06,060 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-11 14:36:06,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-11 14:36:06,062 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:06,063 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c empty. 2023-07-11 14:36:06,064 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:06,064 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-11 14:36:06,091 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-11 14:36:06,092 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => ef703c6f28122a7c70d67fbbd278729c, NAME => 't2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.tmp 2023-07-11 14:36:06,111 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:06,111 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing ef703c6f28122a7c70d67fbbd278729c, disabling compactions & flushes 2023-07-11 14:36:06,111 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:06,111 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:06,111 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. after waiting 0 ms 2023-07-11 14:36:06,111 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:06,112 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:06,112 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for ef703c6f28122a7c70d67fbbd278729c: 2023-07-11 14:36:06,116 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-11 14:36:06,118 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086166118"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689086166118"}]},"ts":"1689086166118"} 2023-07-11 14:36:06,118 INFO [jenkins-hbase9:37647Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:06,120 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-11 14:36:06,122 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-11 14:36:06,122 INFO [jenkins-hbase9:42359Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:06,122 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086166122"}]},"ts":"1689086166122"} 2023-07-11 14:36:06,124 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-11 14:36:06,128 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-07-11 14:36:06,130 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-11 14:36:06,130 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-11 14:36:06,130 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-11 14:36:06,130 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ef703c6f28122a7c70d67fbbd278729c, ASSIGN}] 2023-07-11 14:36:06,133 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ef703c6f28122a7c70d67fbbd278729c, ASSIGN 2023-07-11 14:36:06,134 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=ef703c6f28122a7c70d67fbbd278729c, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,42359,1689086159924; forceNewPlan=false, retain=false 2023-07-11 14:36:06,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-11 14:36:06,285 INFO [jenkins-hbase9:46131] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-11 14:36:06,286 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=ef703c6f28122a7c70d67fbbd278729c, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:06,287 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086166286"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086166286"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086166286"}]},"ts":"1689086166286"} 2023-07-11 14:36:06,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure ef703c6f28122a7c70d67fbbd278729c, server=jenkins-hbase9.apache.org,42359,1689086159924}] 2023-07-11 14:36:06,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-11 14:36:06,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:06,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ef703c6f28122a7c70d67fbbd278729c, NAME => 't2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:06,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:06,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:06,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:06,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:06,451 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:06,452 DEBUG [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/cf1 2023-07-11 14:36:06,452 DEBUG [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/cf1 2023-07-11 14:36:06,453 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ef703c6f28122a7c70d67fbbd278729c columnFamilyName cf1 2023-07-11 14:36:06,453 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] regionserver.HStore(310): Store=ef703c6f28122a7c70d67fbbd278729c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:06,453 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:06,455 DEBUG [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/norep 2023-07-11 14:36:06,455 DEBUG [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/norep 2023-07-11 14:36:06,456 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ef703c6f28122a7c70d67fbbd278729c columnFamilyName norep 2023-07-11 14:36:06,456 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] regionserver.HStore(310): Store=ef703c6f28122a7c70d67fbbd278729c/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:06,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:06,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:06,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-11 14:36:06,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:06,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-11 14:36:06,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened ef703c6f28122a7c70d67fbbd278729c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64942463, jitterRate=-0.032281890511512756}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-11 14:36:06,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for ef703c6f28122a7c70d67fbbd278729c: 2023-07-11 14:36:06,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c., pid=14, masterSystemTime=1689086166443 2023-07-11 14:36:06,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:06,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:06,503 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=ef703c6f28122a7c70d67fbbd278729c, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:06,503 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086166503"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086166503"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086166503"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086166503"}]},"ts":"1689086166503"} 2023-07-11 14:36:06,510 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-11 14:36:06,510 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure ef703c6f28122a7c70d67fbbd278729c, server=jenkins-hbase9.apache.org,42359,1689086159924 in 216 msec 2023-07-11 14:36:06,516 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-11 14:36:06,516 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=ef703c6f28122a7c70d67fbbd278729c, ASSIGN in 380 msec 2023-07-11 14:36:06,521 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-11 14:36:06,521 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086166521"}]},"ts":"1689086166521"} 2023-07-11 14:36:06,524 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-11 14:36:06,531 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-11 14:36:06,533 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 479 msec 2023-07-11 14:36:06,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-11 14:36:06,669 INFO [Listener at localhost/42861] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-11 14:36:06,674 DEBUG [Listener at localhost/42861] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-11 14:36:06,683 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45668, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-11 14:36:06,686 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:06,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-11 14:36:06,693 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-11 14:36:06,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.MasterRpcServices(700): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-11 14:36:06,699 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-11 14:36:06,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-11 14:36:06,705 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/default/t1_syncup/f27301259d11533bab9412d981c5782d 2023-07-11 14:36:06,706 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/default/t1_syncup/f27301259d11533bab9412d981c5782d empty. 2023-07-11 14:36:06,708 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/default/t1_syncup/f27301259d11533bab9412d981c5782d 2023-07-11 14:36:06,708 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-11 14:36:06,744 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-11 14:36:06,746 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => f27301259d11533bab9412d981c5782d, NAME => 't1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp 2023-07-11 14:36:06,766 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:06,766 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing f27301259d11533bab9412d981c5782d, disabling compactions & flushes 2023-07-11 14:36:06,766 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:06,766 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:06,766 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. after waiting 0 ms 2023-07-11 14:36:06,766 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:06,766 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:06,766 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for f27301259d11533bab9412d981c5782d: 2023-07-11 14:36:06,775 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-11 14:36:06,776 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086166776"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689086166776"}]},"ts":"1689086166776"} 2023-07-11 14:36:06,779 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-11 14:36:06,780 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-11 14:36:06,780 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086166780"}]},"ts":"1689086166780"} 2023-07-11 14:36:06,782 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-11 14:36:06,787 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-07-11 14:36:06,788 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-11 14:36:06,788 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-11 14:36:06,788 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-11 14:36:06,788 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-11 14:36:06,788 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-11 14:36:06,788 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN}] 2023-07-11 14:36:06,791 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN 2023-07-11 14:36:06,792 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,42167,1689086163940; forceNewPlan=false, retain=false 2023-07-11 14:36:06,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-11 14:36:06,942 INFO [jenkins-hbase9:34835] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-11 14:36:06,944 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f27301259d11533bab9412d981c5782d, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:06,945 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086166944"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086166944"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086166944"}]},"ts":"1689086166944"} 2023-07-11 14:36:06,948 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure f27301259d11533bab9412d981c5782d, server=jenkins-hbase9.apache.org,42167,1689086163940}] 2023-07-11 14:36:06,987 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-11 14:36:07,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-11 14:36:07,103 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:07,103 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:07,107 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58632, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-11 14:36:07,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:07,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:07,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C42167%2C1689086163940.jenkins-hbase9.apache.org%2C42167%2C1689086163940.regiongroup-1, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:07,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940/jenkins-hbase9.apache.org%2C42167%2C1689086163940.jenkins-hbase9.apache.org%2C42167%2C1689086163940.regiongroup-1.1689086167128 2023-07-11 14:36:07,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK], DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK], DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK]] 2023-07-11 14:36:07,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f27301259d11533bab9412d981c5782d, NAME => 't1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:07,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup f27301259d11533bab9412d981c5782d 2023-07-11 14:36:07,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:07,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:07,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:07,197 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f27301259d11533bab9412d981c5782d 2023-07-11 14:36:07,211 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1 2023-07-11 14:36:07,211 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1 2023-07-11 14:36:07,213 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f27301259d11533bab9412d981c5782d columnFamilyName cf1 2023-07-11 14:36:07,214 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(310): Store=f27301259d11533bab9412d981c5782d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:07,214 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region f27301259d11533bab9412d981c5782d 2023-07-11 14:36:07,217 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/norep 2023-07-11 14:36:07,218 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/norep 2023-07-11 14:36:07,218 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-11 14:36:07,218 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f27301259d11533bab9412d981c5782d columnFamilyName norep 2023-07-11 14:36:07,220 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(310): Store=f27301259d11533bab9412d981c5782d/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:07,220 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-11 14:36:07,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d 2023-07-11 14:36:07,221 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-11 14:36:07,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d 2023-07-11 14:36:07,221 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-11 14:36:07,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-11 14:36:07,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:07,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-11 14:36:07,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened f27301259d11533bab9412d981c5782d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65170633, jitterRate=-0.028881892561912537}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-11 14:36:07,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for f27301259d11533bab9412d981c5782d: 2023-07-11 14:36:07,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d., pid=11, masterSystemTime=1689086167103 2023-07-11 14:36:07,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:07,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:07,237 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f27301259d11533bab9412d981c5782d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:07,237 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086167237"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086167237"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086167237"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086167237"}]},"ts":"1689086167237"} 2023-07-11 14:36:07,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-11 14:36:07,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure f27301259d11533bab9412d981c5782d, server=jenkins-hbase9.apache.org,42167,1689086163940 in 292 msec 2023-07-11 14:36:07,246 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-11 14:36:07,246 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN in 455 msec 2023-07-11 14:36:07,247 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-11 14:36:07,248 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086167248"}]},"ts":"1689086167248"} 2023-07-11 14:36:07,250 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-11 14:36:07,254 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-11 14:36:07,257 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 568 msec 2023-07-11 14:36:07,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-11 14:36:07,307 INFO [Listener at localhost/42861] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-11 14:36:07,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:07,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-11 14:36:07,314 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-11 14:36:07,314 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.MasterRpcServices(700): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-11 14:36:07,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-11 14:36:07,316 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-11 14:36:07,319 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:07,319 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2 empty. 2023-07-11 14:36:07,322 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:07,322 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-11 14:36:07,377 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-11 14:36:07,378 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 34f512aa6fa4c9880527ac501e69b8b2, NAME => 't2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.tmp 2023-07-11 14:36:07,394 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:07,395 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 34f512aa6fa4c9880527ac501e69b8b2, disabling compactions & flushes 2023-07-11 14:36:07,395 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:07,395 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:07,395 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. after waiting 0 ms 2023-07-11 14:36:07,395 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:07,395 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:07,395 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 34f512aa6fa4c9880527ac501e69b8b2: 2023-07-11 14:36:07,398 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-11 14:36:07,400 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086167399"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689086167399"}]},"ts":"1689086167399"} 2023-07-11 14:36:07,402 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-11 14:36:07,404 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-11 14:36:07,404 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086167404"}]},"ts":"1689086167404"} 2023-07-11 14:36:07,407 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-11 14:36:07,412 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-07-11 14:36:07,413 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-11 14:36:07,414 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-11 14:36:07,414 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-11 14:36:07,414 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-11 14:36:07,414 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-11 14:36:07,414 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN}] 2023-07-11 14:36:07,417 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN 2023-07-11 14:36:07,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-11 14:36:07,420 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,42167,1689086163940; forceNewPlan=false, retain=false 2023-07-11 14:36:07,570 INFO [jenkins-hbase9:34835] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-11 14:36:07,572 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=34f512aa6fa4c9880527ac501e69b8b2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:07,572 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086167572"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086167572"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086167572"}]},"ts":"1689086167572"} 2023-07-11 14:36:07,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 34f512aa6fa4c9880527ac501e69b8b2, server=jenkins-hbase9.apache.org,42167,1689086163940}] 2023-07-11 14:36:07,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-11 14:36:07,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:07,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 34f512aa6fa4c9880527ac501e69b8b2, NAME => 't2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:07,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:07,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:07,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:07,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:07,736 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:07,739 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1 2023-07-11 14:36:07,739 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1 2023-07-11 14:36:07,740 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 34f512aa6fa4c9880527ac501e69b8b2 columnFamilyName cf1 2023-07-11 14:36:07,740 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(310): Store=34f512aa6fa4c9880527ac501e69b8b2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:07,740 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:07,742 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/norep 2023-07-11 14:36:07,742 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/norep 2023-07-11 14:36:07,743 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 34f512aa6fa4c9880527ac501e69b8b2 columnFamilyName norep 2023-07-11 14:36:07,743 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(310): Store=34f512aa6fa4c9880527ac501e69b8b2/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:07,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:07,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:07,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-11 14:36:07,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:07,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-11 14:36:07,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 34f512aa6fa4c9880527ac501e69b8b2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64173770, jitterRate=-0.04373630881309509}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-11 14:36:07,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 34f512aa6fa4c9880527ac501e69b8b2: 2023-07-11 14:36:07,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2., pid=14, masterSystemTime=1689086167729 2023-07-11 14:36:07,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:07,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:07,762 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=34f512aa6fa4c9880527ac501e69b8b2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:07,762 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086167761"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086167761"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086167761"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086167761"}]},"ts":"1689086167761"} 2023-07-11 14:36:07,769 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-11 14:36:07,769 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 34f512aa6fa4c9880527ac501e69b8b2, server=jenkins-hbase9.apache.org,42167,1689086163940 in 190 msec 2023-07-11 14:36:07,773 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-11 14:36:07,773 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN in 355 msec 2023-07-11 14:36:07,774 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-11 14:36:07,774 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689086167774"}]},"ts":"1689086167774"} 2023-07-11 14:36:07,776 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-11 14:36:07,780 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-11 14:36:07,783 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 471 msec 2023-07-11 14:36:07,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-11 14:36:07,922 INFO [Listener at localhost/42861] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-11 14:36:07,922 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(139): Connect 0x709fcb85 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:07,939 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6f3c7054, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:07,940 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(139): Connect 0x2a94bf46 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:07,954 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24598a72, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:07,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.HMaster(3464): Client=jenkins//172.31.2.10 creating replication peer, id=1, config=clusterKey=127.0.0.1:64930:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-07-11 14:36:07,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-11 14:36:07,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-11 14:36:08,004 INFO [PEWorker-1] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-11 14:36:08,005 INFO [PEWorker-1] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@379b129check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:08,012 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@379b129check-peer-cluster-id0x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:08,014 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@379b129check-peer-cluster-id-0x10154c21b790014 connected 2023-07-11 14:36:08,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-11 14:36:08,119 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@379b129check-peer-cluster-id-0x10154c21b790014, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:08,133 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:08,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-07-11 14:36:08,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-11 14:36:08,318 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:08,318 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:08,323 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42359] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-07-11 14:36:08,334 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-11 14:36:08,342 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53698, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-11 14:36:08,343 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37647] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-07-11 14:36:08,345 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-11 14:36:08,357 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-11 14:36:08,358 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-11 14:36:08,382 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-11 14:36:08,386 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-11 14:36:08,422 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-07-11 14:36:08,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.HMaster(3641): Remote procedure done, pid=16 2023-07-11 14:36:08,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase9.apache.org,37647,1689086159883 suceeded 2023-07-11 14:36:08,430 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-07-11 14:36:08,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.HMaster(3641): Remote procedure done, pid=17 2023-07-11 14:36:08,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase9.apache.org,42359,1689086159924 suceeded 2023-07-11 14:36:08,435 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 284 msec 2023-07-11 14:36:08,438 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-07-11 14:36:08,438 INFO [PEWorker-4] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:64930:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-07-11 14:36:08,438 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 288 msec 2023-07-11 14:36:08,440 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 463 msec 2023-07-11 14:36:08,460 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x078af769 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:08,464 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x3a9a42b6 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:08,472 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1dddcf8a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:08,472 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5495ead4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:08,477 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:08,477 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:08,484 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:08,485 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10154c21b790017 connected 2023-07-11 14:36:08,486 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:08,488 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10154c21b790018 connected 2023-07-11 14:36:08,488 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=1) is replicating from cluster=fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 to cluster=6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:08,488 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 to cluster=6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:08,490 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0 2023-07-11 14:36:08,490 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1 2023-07-11 14:36:08,496 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-11 14:36:08,496 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-11 14:36:08,498 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0 2023-07-11 14:36:08,498 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0 2023-07-11 14:36:08,498 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-11 14:36:08,498 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1 2023-07-11 14:36:08,502 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0 2023-07-11 14:36:08,581 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:08,588 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-11 14:36:08,588 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-11 14:36:08,588 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-11 14:36:08,588 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-11 14:36:08,588 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-11 14:36:08,589 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-11 14:36:08,589 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-11 14:36:08,589 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154, lastWalPosition=901, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-11 14:36:08,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46131] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-11 14:36:08,594 DEBUG [Listener at localhost/42861] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-07-11 14:36:08,596 DEBUG [hconnection-0x4d86f62f-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:08,599 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:50546, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:08,694 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:08,704 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-11 14:36:08,713 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 5614 2023-07-11 14:36:08,714 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 50 WAL entries eligible for replication 2023-07-11 14:36:08,714 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/3bfd16063a0672430ddd816d03e73f73/4=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/5=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/6=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/7=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/8=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/9=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/10=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/11=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/12=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/13=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/14=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/15=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/16=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/17=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/18=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/19=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/20=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/21=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/22=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/23=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/24=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/25=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/26=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/27=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/28=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/29=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/30=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/31=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/32=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/33=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/34=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/35=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/36=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/37=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/38=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/39=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/40=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/41=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/42=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/43=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/44=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/45=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/46=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/47=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/48=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/49=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/50=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/51=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/52=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/53=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237, lastWalPosition=5614, nbRowKeys=50, nbHFiles=0, heapSize=10400, lastSeqIds={}, endOfFile=false,usedBufferSize=10400] 2023-07-11 14:36:08,715 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 5614 2023-07-11 14:36:08,734 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:08,750 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:44774, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-11 14:36:08,755 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34239] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-11 14:36:08,756 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34239] zookeeper.ReadOnlyZKClient(139): Connect 0x5f4a23c8 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:08,765 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34239] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@12d090a9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:08,776 DEBUG [hconnection-0xec1b331-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:08,779 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55870, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-11 14:36:08,808 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:08,826 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-11 14:36:08,834 DEBUG [hconnection-0xec1b331-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:08,838 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58636, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-11 14:36:08,846 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10549 2023-07-11 14:36:08,846 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 47 WAL entries eligible for replication 2023-07-11 14:36:08,847 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10549 2023-07-11 14:36:08,870 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34239] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-11 14:36:08,875 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/3bfd16063a0672430ddd816d03e73f73/54=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/55=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/56=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/57=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/58=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/59=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/60=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/61=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/62=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/63=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/64=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/65=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/66=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/67=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/68=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/69=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/70=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/71=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/72=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/73=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/74=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/75=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/76=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/77=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/78=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/79=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/80=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/81=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/82=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/83=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/84=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/85=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/86=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/87=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/88=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/89=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/90=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/91=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/92=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/93=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/94=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/95=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/96=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/97=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/98=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/99=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/100=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237, lastWalPosition=10549, nbRowKeys=47, nbHFiles=0, heapSize=9776, lastSeqIds={}, endOfFile=false,usedBufferSize=9776] 2023-07-11 14:36:08,879 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:08,880 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-11 14:36:08,881 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-11 14:36:08,891 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58648, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-11 14:36:08,892 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42167] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-11 14:36:08,893 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42167] zookeeper.ReadOnlyZKClient(139): Connect 0x51a54fc8 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:08,902 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42167] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@32dbb05c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:08,923 DEBUG [hconnection-0x60bedc29-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:08,930 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55876, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-11 14:36:08,935 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:08,955 DEBUG [hconnection-0x60bedc29-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:08,968 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58650, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-11 14:36:08,974 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:08,975 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 3 WAL entries eligible for replication 2023-07-11 14:36:08,975 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:08,977 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 4451 2023-07-11 14:36:08,978 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 34 WAL entries eligible for replication 2023-07-11 14:36:08,978 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 4451 2023-07-11 14:36:08,978 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/ef703c6f28122a7c70d67fbbd278729c/4=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/5=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/6=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/7=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/8=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/9=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/10=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/11=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/12=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/13=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/14=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/15=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/16=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/17=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/18=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/19=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/20=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/21=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/22=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/23=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/24=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/25=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/26=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/27=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/28=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/29=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/30=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/31=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/32=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/33=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/34=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/35=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/36=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/37=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154, lastWalPosition=4451, nbRowKeys=34, nbHFiles=0, heapSize=7072, lastSeqIds={}, endOfFile=false,usedBufferSize=7072] 2023-07-11 14:36:08,979 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42167] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-11 14:36:08,980 DEBUG [SinkThread-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:08,984 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/3bfd16063a0672430ddd816d03e73f73/101=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/102=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237, lastWalPosition=10975, nbRowKeys=3, nbHFiles=0, heapSize=624, lastSeqIds={}, endOfFile=false,usedBufferSize=624] 2023-07-11 14:36:08,984 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:41548, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-11 14:36:08,987 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37977] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-11 14:36:08,987 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37977] zookeeper.ReadOnlyZKClient(139): Connect 0x64b06efe to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:08,989 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=37977] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-11 14:36:08,997 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37977] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@6a778c18, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:09,003 DEBUG [hconnection-0x6e48058e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:09,004 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55878, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-11 14:36:09,020 DEBUG [hconnection-0x6e48058e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:09,021 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58666, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-11 14:36:09,024 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=37977] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-11 14:36:09,033 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37977] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-11 14:36:09,047 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:09,089 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:09,112 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10121 2023-07-11 14:36:09,113 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 54 WAL entries eligible for replication 2023-07-11 14:36:09,113 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10121 2023-07-11 14:36:09,113 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/ef703c6f28122a7c70d67fbbd278729c/38=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/39=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/40=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/41=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/42=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/43=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/44=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/45=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/46=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/47=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/48=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/49=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/50=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/51=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/52=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/53=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/54=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/55=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/56=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/57=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/58=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/59=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/60=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/61=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/62=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/63=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/64=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/65=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/66=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/67=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/68=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/69=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/70=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/71=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/72=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/73=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/74=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/75=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/76=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/77=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/78=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/79=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/80=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/81=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/82=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/83=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/84=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/85=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/86=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/87=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/88=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/89=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/90=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/91=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154, lastWalPosition=10121, nbRowKeys=54, nbHFiles=0, heapSize=11232, lastSeqIds={}, endOfFile=false,usedBufferSize=11232] 2023-07-11 14:36:09,116 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34239] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-11 14:36:09,127 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34239] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-11 14:36:09,160 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:09,212 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:09,235 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 16973 2023-07-11 14:36:09,235 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 64 WAL entries eligible for replication 2023-07-11 14:36:09,235 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 16973 2023-07-11 14:36:09,236 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/ef703c6f28122a7c70d67fbbd278729c/92=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/93=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/94=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/95=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/96=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/97=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/98=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/99=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/100=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/101=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/102=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/103=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/104=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/105=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/106=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/107=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/108=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/109=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/110=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/111=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/112=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/113=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/114=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/115=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/116=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/117=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/118=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/119=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/120=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/121=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/122=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/123=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/124=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/125=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/126=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/127=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/128=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/129=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/130=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/131=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/132=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/133=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/134=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/135=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/136=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/137=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/138=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/139=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/140=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/141=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/142=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/143=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/144=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/145=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/146=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/147=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/148=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/149=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/150=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/151=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/152=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/153=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/154=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/155=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154, lastWalPosition=16973, nbRowKeys=64, nbHFiles=0, heapSize=13312, lastSeqIds={}, endOfFile=false,usedBufferSize=13312] 2023-07-11 14:36:09,242 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37977] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-11 14:36:09,251 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37977] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-11 14:36:09,280 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:09,328 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:09,350 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:09,350 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 48 WAL entries eligible for replication 2023-07-11 14:36:09,350 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:09,351 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/ef703c6f28122a7c70d67fbbd278729c/156=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/157=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/158=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/159=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/160=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/161=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/162=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/163=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/164=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/165=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/166=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/167=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/168=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/169=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/170=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/171=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/172=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/173=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/174=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/175=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/176=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/177=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/178=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/179=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/180=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/181=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/182=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/183=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/184=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/185=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/186=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/187=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/188=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/189=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/190=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/191=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/192=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/193=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/194=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/195=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/196=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/197=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/198=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/199=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/200=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/201=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/202=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154, lastWalPosition=22269, nbRowKeys=48, nbHFiles=0, heapSize=9984, lastSeqIds={}, endOfFile=false,usedBufferSize=9984] 2023-07-11 14:36:09,353 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37977] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-11 14:36:09,358 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37977] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-11 14:36:09,388 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:09,438 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:09,456 INFO [jenkins-hbase9:41517Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:09,458 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:09,466 INFO [jenkins-hbase9:42167Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:09,466 INFO [jenkins-hbase9:42167Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 47 2023-07-11 14:36:09,466 INFO [jenkins-hbase9:34239Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:09,466 INFO [jenkins-hbase9:34239Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 104 2023-07-11 14:36:09,466 INFO [jenkins-hbase9:37977Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:09,467 INFO [jenkins-hbase9:37977Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 149 2023-07-11 14:36:09,497 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:09,547 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:09,565 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:09,604 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:09,655 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:09,672 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:09,711 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:09,769 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:09,778 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:09,818 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:09,839 DEBUG [hconnection-0x286bf07d-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:09,843 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55890, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:09,848 DEBUG [hconnection-0x286bf07d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:09,852 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58680, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:09,877 DEBUG [Listener at localhost/42861] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-07-11 14:36:09,877 INFO [Listener at localhost/42861] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-11 14:36:09,877 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1ac7aba1 to 127.0.0.1:64930 2023-07-11 14:36:09,877 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:09,878 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:09,878 DEBUG [Listener at localhost/42861] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-11 14:36:09,878 DEBUG [Listener at localhost/42861] util.JVMClusterUtil(257): Found active master hash=560046124, stopped=false 2023-07-11 14:36:09,878 INFO [Listener at localhost/42861] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:09,882 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-11 14:36:09,882 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-11 14:36:09,882 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:09,882 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-11 14:36:09,882 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-11 14:36:09,883 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:09,883 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:09,883 INFO [Listener at localhost/42861] procedure2.ProcedureExecutor(629): Stopping 2023-07-11 14:36:09,883 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-11 14:36:09,883 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:09,883 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6f08f31c to 127.0.0.1:64930 2023-07-11 14:36:09,883 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:09,885 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1064): Closing user regions 2023-07-11 14:36:09,885 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(3305): Received CLOSE for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:09,885 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:09,886 INFO [Listener at localhost/42861] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,37977,1689086163856' ***** 2023-07-11 14:36:09,886 INFO [Listener at localhost/42861] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-11 14:36:09,886 INFO [Listener at localhost/42861] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,41517,1689086163885' ***** 2023-07-11 14:36:09,886 INFO [Listener at localhost/42861] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-11 14:36:09,886 INFO [Listener at localhost/42861] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,34239,1689086163911' ***** 2023-07-11 14:36:09,886 INFO [Listener at localhost/42861] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-11 14:36:09,886 INFO [Listener at localhost/42861] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,42167,1689086163940' ***** 2023-07-11 14:36:09,886 INFO [Listener at localhost/42861] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-11 14:36:09,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:09,893 INFO [RS:2;jenkins-hbase9:34239] regionserver.HeapMemoryManager(220): Stopping 2023-07-11 14:36:09,893 INFO [RS:2;jenkins-hbase9:34239] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-11 14:36:09,895 INFO [RS:2;jenkins-hbase9:34239] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-11 14:36:09,895 INFO [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:09,895 DEBUG [RS:2;jenkins-hbase9:34239] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4c0cc103 to 127.0.0.1:64930 2023-07-11 14:36:09,895 DEBUG [RS:2;jenkins-hbase9:34239] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:09,895 INFO [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,34239,1689086163911; all regions closed. 2023-07-11 14:36:09,895 INFO [RS:3;jenkins-hbase9:42167] regionserver.HeapMemoryManager(220): Stopping 2023-07-11 14:36:09,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing d4185bdf44467f073faaa5de49a197d7, disabling compactions & flushes 2023-07-11 14:36:09,896 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-11 14:36:09,897 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:09,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:09,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. after waiting 0 ms 2023-07-11 14:36:09,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:09,897 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:09,901 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-11 14:36:09,901 INFO [RS:1;jenkins-hbase9:41517] regionserver.HeapMemoryManager(220): Stopping 2023-07-11 14:36:09,901 INFO [RS:1;jenkins-hbase9:41517] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-11 14:36:09,901 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-11 14:36:09,901 INFO [RS:1;jenkins-hbase9:41517] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-11 14:36:09,901 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:09,902 DEBUG [RS:1;jenkins-hbase9:41517] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x48790efc to 127.0.0.1:64930 2023-07-11 14:36:09,902 DEBUG [RS:1;jenkins-hbase9:41517] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:09,902 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing d4185bdf44467f073faaa5de49a197d7 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-11 14:36:09,902 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:09,903 INFO [RS:3;jenkins-hbase9:42167] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-11 14:36:09,903 INFO [RS:0;jenkins-hbase9:37977] regionserver.HeapMemoryManager(220): Stopping 2023-07-11 14:36:09,902 INFO [RS:1;jenkins-hbase9:41517] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-11 14:36:09,906 INFO [RS:0;jenkins-hbase9:37977] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-11 14:36:09,906 INFO [RS:1;jenkins-hbase9:41517] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-11 14:36:09,906 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-11 14:36:09,906 INFO [RS:3;jenkins-hbase9:42167] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-11 14:36:09,906 INFO [RS:1;jenkins-hbase9:41517] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-11 14:36:09,906 INFO [RS:0;jenkins-hbase9:37977] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-11 14:36:09,907 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-11 14:36:09,907 INFO [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:09,907 INFO [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(3305): Received CLOSE for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:09,907 DEBUG [RS:0;jenkins-hbase9:37977] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3673105a to 127.0.0.1:64930 2023-07-11 14:36:09,908 DEBUG [RS:0;jenkins-hbase9:37977] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:09,908 INFO [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,37977,1689086163856; all regions closed. 2023-07-11 14:36:09,908 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-11 14:36:09,908 DEBUG [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, d4185bdf44467f073faaa5de49a197d7=hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.} 2023-07-11 14:36:09,908 INFO [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(3305): Received CLOSE for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:09,908 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:09,908 INFO [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:09,908 DEBUG [RS:3;jenkins-hbase9:42167] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x22ac3689 to 127.0.0.1:64930 2023-07-11 14:36:09,908 DEBUG [RS:3;jenkins-hbase9:42167] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:09,909 INFO [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-11 14:36:09,909 DEBUG [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1478): Online Regions={f27301259d11533bab9412d981c5782d=t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d., 34f512aa6fa4c9880527ac501e69b8b2=t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.} 2023-07-11 14:36:09,910 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-11 14:36:09,910 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-11 14:36:09,910 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-11 14:36:09,911 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-11 14:36:09,911 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-11 14:36:09,911 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-07-11 14:36:09,911 DEBUG [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1504): Waiting on 34f512aa6fa4c9880527ac501e69b8b2, f27301259d11533bab9412d981c5782d 2023-07-11 14:36:09,911 DEBUG [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1504): Waiting on 1588230740, d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:09,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing f27301259d11533bab9412d981c5782d, disabling compactions & flushes 2023-07-11 14:36:09,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:09,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:09,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. after waiting 0 ms 2023-07-11 14:36:09,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:09,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing f27301259d11533bab9412d981c5782d 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-11 14:36:09,923 DEBUG [RS:2;jenkins-hbase9:34239] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:36:09,923 DEBUG [RS:0;jenkins-hbase9:37977] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:36:09,923 INFO [RS:0;jenkins-hbase9:37977] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C37977%2C1689086163856.jenkins-hbase9.apache.org%2C37977%2C1689086163856.regiongroup-0:(num 1689086164589) 2023-07-11 14:36:09,923 DEBUG [RS:0;jenkins-hbase9:37977] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:09,923 INFO [RS:0;jenkins-hbase9:37977] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:09,923 INFO [RS:2;jenkins-hbase9:34239] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C34239%2C1689086163911.jenkins-hbase9.apache.org%2C34239%2C1689086163911.regiongroup-0:(num 1689086164589) 2023-07-11 14:36:09,924 DEBUG [RS:2;jenkins-hbase9:34239] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:09,924 INFO [RS:2;jenkins-hbase9:34239] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:09,925 INFO [RS:2;jenkins-hbase9:34239] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-11 14:36:09,925 INFO [RS:0;jenkins-hbase9:37977] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-11 14:36:09,925 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:09,926 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:09,926 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:09,927 INFO [RS:2;jenkins-hbase9:34239] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-11 14:36:09,927 INFO [RS:2;jenkins-hbase9:34239] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-11 14:36:09,927 INFO [RS:2;jenkins-hbase9:34239] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-11 14:36:09,926 INFO [RS:0;jenkins-hbase9:37977] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-11 14:36:09,927 INFO [RS:0;jenkins-hbase9:37977] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-11 14:36:09,928 INFO [RS:0;jenkins-hbase9:37977] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-11 14:36:09,928 DEBUG [RS:0;jenkins-hbase9:37977] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64b06efe to 127.0.0.1:64930 2023-07-11 14:36:09,928 DEBUG [RS:0;jenkins-hbase9:37977] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:09,928 DEBUG [RS:2;jenkins-hbase9:34239] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f4a23c8 to 127.0.0.1:64930 2023-07-11 14:36:09,928 DEBUG [RS:2;jenkins-hbase9:34239] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:09,930 INFO [RS:2;jenkins-hbase9:34239] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:34239 2023-07-11 14:36:09,930 INFO [RS:0;jenkins-hbase9:37977] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:37977 2023-07-11 14:36:09,930 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:09,942 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:09,949 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:09,949 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:09,949 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:09,949 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:09,950 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:09,950 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:09,949 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:09,951 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b790018, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:09,951 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:09,951 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-11 14:36:09,951 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,37977,1689086163856 2023-07-11 14:36:09,950 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:09,951 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:09,951 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,37977,1689086163856] 2023-07-11 14:36:09,951 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:09,951 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:09,951 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,37977,1689086163856; numProcessing=1 2023-07-11 14:36:09,951 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,34239,1689086163911 2023-07-11 14:36:09,953 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,37977,1689086163856 already deleted, retry=false 2023-07-11 14:36:09,953 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,37977,1689086163856 expired; onlineServers=3 2023-07-11 14:36:09,953 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,34239,1689086163911] 2023-07-11 14:36:09,953 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,34239,1689086163911; numProcessing=2 2023-07-11 14:36:09,955 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,34239,1689086163911 already deleted, retry=false 2023-07-11 14:36:09,955 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,34239,1689086163911 expired; onlineServers=2 2023-07-11 14:36:09,962 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:09,989 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:10,016 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:10,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/.tmp/info/ca4f495a328442daad2ea4bf64ee841b 2023-07-11 14:36:10,030 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/.tmp/info/2023d570cfaa402593275b0451eb9042 2023-07-11 14:36:10,030 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/.tmp/cf1/37e3e6e8b34041bd8bc6a3e3df3c42bd 2023-07-11 14:36:10,045 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:10,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/.tmp/info/ca4f495a328442daad2ea4bf64ee841b as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/info/ca4f495a328442daad2ea4bf64ee841b 2023-07-11 14:36:10,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/.tmp/cf1/37e3e6e8b34041bd8bc6a3e3df3c42bd as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1/37e3e6e8b34041bd8bc6a3e3df3c42bd 2023-07-11 14:36:10,082 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:10,082 INFO [RS:2;jenkins-hbase9:34239] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,34239,1689086163911; zookeeper connection closed. 2023-07-11 14:36:10,082 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:34239-0x10154c21b79000a, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:10,085 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@67bedd60] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@67bedd60 2023-07-11 14:36:10,089 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1/37e3e6e8b34041bd8bc6a3e3df3c42bd, entries=100, sequenceid=7, filesize=8.3 K 2023-07-11 14:36:10,089 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/info/ca4f495a328442daad2ea4bf64ee841b, entries=2, sequenceid=6, filesize=4.8 K 2023-07-11 14:36:10,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for f27301259d11533bab9412d981c5782d in 179ms, sequenceid=7, compaction requested=false 2023-07-11 14:36:10,094 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for d4185bdf44467f073faaa5de49a197d7 in 192ms, sequenceid=6, compaction requested=false 2023-07-11 14:36:10,098 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/.tmp/table/ed3b6a6f597941748150410cb9e7546c 2023-07-11 14:36:10,101 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:10,109 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-11 14:36:10,109 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-11 14:36:10,110 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:10,111 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:10,111 DEBUG [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1504): Waiting on 34f512aa6fa4c9880527ac501e69b8b2, f27301259d11533bab9412d981c5782d 2023-07-11 14:36:10,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for d4185bdf44467f073faaa5de49a197d7: 2023-07-11 14:36:10,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for f27301259d11533bab9412d981c5782d: 2023-07-11 14:36:10,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:10,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:10,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 34f512aa6fa4c9880527ac501e69b8b2, disabling compactions & flushes 2023-07-11 14:36:10,113 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/.tmp/info/2023d570cfaa402593275b0451eb9042 as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info/2023d570cfaa402593275b0451eb9042 2023-07-11 14:36:10,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:10,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:10,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. after waiting 0 ms 2023-07-11 14:36:10,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:10,113 DEBUG [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-11 14:36:10,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 34f512aa6fa4c9880527ac501e69b8b2 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-11 14:36:10,120 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info/2023d570cfaa402593275b0451eb9042, entries=30, sequenceid=19, filesize=8.1 K 2023-07-11 14:36:10,122 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/.tmp/table/ed3b6a6f597941748150410cb9e7546c as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table/ed3b6a6f597941748150410cb9e7546c 2023-07-11 14:36:10,124 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:10,130 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table/ed3b6a6f597941748150410cb9e7546c, entries=6, sequenceid=19, filesize=4.8 K 2023-07-11 14:36:10,131 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 220ms, sequenceid=19, compaction requested=false 2023-07-11 14:36:10,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=8 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/.tmp/cf1/e3e0f680c76e48bf80b60121dfd11e86 2023-07-11 14:36:10,144 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-11 14:36:10,145 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-11 14:36:10,146 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-11 14:36:10,146 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-11 14:36:10,146 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-11 14:36:10,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/.tmp/cf1/e3e0f680c76e48bf80b60121dfd11e86 as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1/e3e0f680c76e48bf80b60121dfd11e86 2023-07-11 14:36:10,150 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:10,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1/e3e0f680c76e48bf80b60121dfd11e86, entries=200, sequenceid=8, filesize=12.2 K 2023-07-11 14:36:10,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 34f512aa6fa4c9880527ac501e69b8b2 in 42ms, sequenceid=8, compaction requested=false 2023-07-11 14:36:10,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/recovered.edits/11.seqid, newMaxSeqId=11, maxSeqId=1 2023-07-11 14:36:10,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:10,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 34f512aa6fa4c9880527ac501e69b8b2: 2023-07-11 14:36:10,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:10,182 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:10,182 INFO [RS:0;jenkins-hbase9:37977] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,37977,1689086163856; zookeeper connection closed. 2023-07-11 14:36:10,182 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:37977-0x10154c21b790008, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:10,183 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@b840f40] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@b840f40 2023-07-11 14:36:10,208 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:10,229 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:10,256 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:10,312 INFO [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,42167,1689086163940; all regions closed. 2023-07-11 14:36:10,312 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:10,314 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,41517,1689086163885; all regions closed. 2023-07-11 14:36:10,315 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:10,326 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:10,331 DEBUG [RS:1;jenkins-hbase9:41517] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:36:10,331 INFO [RS:1;jenkins-hbase9:41517] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C41517%2C1689086163885.meta:.meta(num 1689086164679) 2023-07-11 14:36:10,331 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:10,335 DEBUG [RS:3;jenkins-hbase9:42167] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:36:10,335 INFO [RS:3;jenkins-hbase9:42167] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C42167%2C1689086163940.jenkins-hbase9.apache.org%2C42167%2C1689086163940.regiongroup-1:(num 1689086167128) 2023-07-11 14:36:10,337 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:10,338 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:10,347 DEBUG [RS:1;jenkins-hbase9:41517] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:36:10,347 INFO [RS:1;jenkins-hbase9:41517] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C41517%2C1689086163885.jenkins-hbase9.apache.org%2C41517%2C1689086163885.regiongroup-1:(num 1689086165150) 2023-07-11 14:36:10,347 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:10,352 DEBUG [RS:3;jenkins-hbase9:42167] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:36:10,352 INFO [RS:3;jenkins-hbase9:42167] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C42167%2C1689086163940.jenkins-hbase9.apache.org%2C42167%2C1689086163940.regiongroup-0:(num 1689086164588) 2023-07-11 14:36:10,352 DEBUG [RS:3;jenkins-hbase9:42167] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:10,352 INFO [RS:3;jenkins-hbase9:42167] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:10,354 INFO [RS:3;jenkins-hbase9:42167] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-11 14:36:10,354 INFO [RS:3;jenkins-hbase9:42167] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-11 14:36:10,354 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:10,354 INFO [RS:3;jenkins-hbase9:42167] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-11 14:36:10,354 INFO [RS:3;jenkins-hbase9:42167] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-11 14:36:10,354 DEBUG [RS:3;jenkins-hbase9:42167] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x51a54fc8 to 127.0.0.1:64930 2023-07-11 14:36:10,355 DEBUG [RS:3;jenkins-hbase9:42167] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:10,355 INFO [RS:3;jenkins-hbase9:42167] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:42167 2023-07-11 14:36:10,362 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:10,362 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:10,362 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b790018, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:10,362 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-11 14:36:10,362 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,42167,1689086163940] 2023-07-11 14:36:10,362 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,42167,1689086163940; numProcessing=3 2023-07-11 14:36:10,363 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,42167,1689086163940 2023-07-11 14:36:10,364 DEBUG [RS:1;jenkins-hbase9:41517] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:36:10,364 INFO [RS:1;jenkins-hbase9:41517] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C41517%2C1689086163885.jenkins-hbase9.apache.org%2C41517%2C1689086163885.regiongroup-0:(num 1689086164566) 2023-07-11 14:36:10,364 DEBUG [RS:1;jenkins-hbase9:41517] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:10,364 INFO [RS:1;jenkins-hbase9:41517] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:10,364 INFO [RS:1;jenkins-hbase9:41517] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-11 14:36:10,364 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:10,365 INFO [RS:1;jenkins-hbase9:41517] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:41517 2023-07-11 14:36:10,365 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,42167,1689086163940 already deleted, retry=false 2023-07-11 14:36:10,365 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,42167,1689086163940 expired; onlineServers=1 2023-07-11 14:36:10,367 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,41517,1689086163885 2023-07-11 14:36:10,367 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b790018, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:10,367 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:10,368 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-11 14:36:10,368 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,41517,1689086163885] 2023-07-11 14:36:10,368 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,41517,1689086163885; numProcessing=4 2023-07-11 14:36:10,368 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:10,369 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,41517,1689086163885 already deleted, retry=false 2023-07-11 14:36:10,369 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,41517,1689086163885 expired; onlineServers=0 2023-07-11 14:36:10,369 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,34835,1689086163812' ***** 2023-07-11 14:36:10,369 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-11 14:36:10,370 DEBUG [M:0;jenkins-hbase9:34835] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@255e30d8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:10,370 INFO [M:0;jenkins-hbase9:34835] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:10,370 INFO [M:0;jenkins-hbase9:34835] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,34835,1689086163812; all regions closed. 2023-07-11 14:36:10,370 DEBUG [M:0;jenkins-hbase9:34835] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:10,370 DEBUG [M:0;jenkins-hbase9:34835] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-11 14:36:10,371 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-11 14:36:10,371 DEBUG [M:0;jenkins-hbase9:34835] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-11 14:36:10,371 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086164203] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086164203,5,FailOnTimeoutGroup] 2023-07-11 14:36:10,371 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086164206] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086164206,5,FailOnTimeoutGroup] 2023-07-11 14:36:10,372 INFO [M:0;jenkins-hbase9:34835] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-11 14:36:10,373 INFO [M:0;jenkins-hbase9:34835] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-11 14:36:10,373 INFO [M:0;jenkins-hbase9:34835] hbase.ChoreService(369): Chore service for: master/jenkins-hbase9:0 had [] on shutdown 2023-07-11 14:36:10,373 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-11 14:36:10,373 DEBUG [M:0;jenkins-hbase9:34835] master.HMaster(1512): Stopping service threads 2023-07-11 14:36:10,373 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:10,373 INFO [M:0;jenkins-hbase9:34835] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-11 14:36:10,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:10,374 ERROR [M:0;jenkins-hbase9:34835] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-11 14:36:10,375 INFO [M:0;jenkins-hbase9:34835] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-11 14:36:10,375 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-11 14:36:10,375 DEBUG [M:0;jenkins-hbase9:34835] zookeeper.ZKUtil(398): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-11 14:36:10,375 WARN [M:0;jenkins-hbase9:34835] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-11 14:36:10,375 INFO [M:0;jenkins-hbase9:34835] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-11 14:36:10,376 INFO [M:0;jenkins-hbase9:34835] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-11 14:36:10,377 DEBUG [M:0;jenkins-hbase9:34835] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-11 14:36:10,377 INFO [M:0;jenkins-hbase9:34835] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:10,377 DEBUG [M:0;jenkins-hbase9:34835] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:10,377 DEBUG [M:0;jenkins-hbase9:34835] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-11 14:36:10,377 DEBUG [M:0;jenkins-hbase9:34835] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:10,377 INFO [M:0;jenkins-hbase9:34835] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.40 KB heapSize=65.18 KB 2023-07-11 14:36:10,390 INFO [M:0;jenkins-hbase9:34835] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.40 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/bafe90e861e748fda9018cd963db289a 2023-07-11 14:36:10,398 DEBUG [M:0;jenkins-hbase9:34835] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/bafe90e861e748fda9018cd963db289a as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bafe90e861e748fda9018cd963db289a 2023-07-11 14:36:10,403 INFO [M:0;jenkins-hbase9:34835] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bafe90e861e748fda9018cd963db289a, entries=14, sequenceid=116, filesize=8.1 K 2023-07-11 14:36:10,405 INFO [M:0;jenkins-hbase9:34835] regionserver.HRegion(2948): Finished flush of dataSize ~55.40 KB/56732, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 27ms, sequenceid=116, compaction requested=false 2023-07-11 14:36:10,407 INFO [M:0;jenkins-hbase9:34835] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:10,407 DEBUG [M:0;jenkins-hbase9:34835] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:10,407 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,34835,1689086163812 2023-07-11 14:36:10,414 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:10,414 INFO [M:0;jenkins-hbase9:34835] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-11 14:36:10,415 INFO [M:0;jenkins-hbase9:34835] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:34835 2023-07-11 14:36:10,417 DEBUG [M:0;jenkins-hbase9:34835] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase9.apache.org,34835,1689086163812 already deleted, retry=false 2023-07-11 14:36:10,432 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:10,442 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:10,473 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:10,537 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:10,548 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:10,579 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:10,643 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:10,654 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:10,687 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:10,752 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:10,759 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:10,793 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:10,859 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:10,864 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:10,898 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:10,965 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:10,969 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:10,984 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:10,984 INFO [M:0;jenkins-hbase9:34835] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,34835,1689086163812; zookeeper connection closed. 2023-07-11 14:36:10,984 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:34835-0x10154c21b790007, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:11,003 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:11,072 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:11,075 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:11,084 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:11,084 INFO [RS:1;jenkins-hbase9:41517] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,41517,1689086163885; zookeeper connection closed. 2023-07-11 14:36:11,084 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:41517-0x10154c21b790009, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:11,086 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@62a2acec] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@62a2acec 2023-07-11 14:36:11,109 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:11,118 INFO [jenkins-hbase9:37647Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0]: currently replicating from: hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,37647,1689086159883/jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0.1689086161243 at position: -1 2023-07-11 14:36:11,122 INFO [jenkins-hbase9:42359Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 300, current progress: walGroup [jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1]: currently replicating from: hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154 at position: 22269 walGroup [jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0]: currently replicating from: hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237 at position: 10975 2023-07-11 14:36:11,178 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-11 14:36:11,180 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-11 14:36:11,184 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:11,184 INFO [RS:3;jenkins-hbase9:42167] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,42167,1689086163940; zookeeper connection closed. 2023-07-11 14:36:11,185 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42167-0x10154c21b79000b, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:11,186 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@74df9ec2] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@74df9ec2 2023-07-11 14:36:11,186 INFO [Listener at localhost/42861] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-07-11 14:36:11,210 INFO [Listener at localhost/42861] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-11 14:36:11,210 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5003310f to 127.0.0.1:64930 2023-07-11 14:36:11,211 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:11,211 DEBUG [Listener at localhost/42861] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-11 14:36:11,211 DEBUG [Listener at localhost/42861] util.JVMClusterUtil(257): Found active master hash=1320483604, stopped=false 2023-07-11 14:36:11,211 INFO [Listener at localhost/42861] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:36:11,214 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-11 14:36:11,214 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-11 14:36:11,214 INFO [Listener at localhost/42861] procedure2.ProcedureExecutor(629): Stopping 2023-07-11 14:36:11,214 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-11 14:36:11,217 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:11,217 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x007d23e9 to 127.0.0.1:64930 2023-07-11 14:36:11,217 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:11,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-11 14:36:11,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-11 14:36:11,286 INFO [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,37647,1689086159883' ***** 2023-07-11 14:36:11,286 INFO [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(2311): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-07-11 14:36:11,217 INFO [Listener at localhost/42861] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,37647,1689086159883' ***** 2023-07-11 14:36:11,286 INFO [Listener at localhost/42861] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-11 14:36:11,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-11 14:36:11,286 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1064): Closing user regions 2023-07-11 14:36:11,292 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(3305): Received CLOSE for 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:11,292 INFO [Listener at localhost/42861] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,42359,1689086159924' ***** 2023-07-11 14:36:11,293 INFO [Listener at localhost/42861] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-11 14:36:11,293 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(3305): Received CLOSE for 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:11,293 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(3305): Received CLOSE for ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:11,294 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:11,295 INFO [RS:1;jenkins-hbase9:42359] regionserver.HeapMemoryManager(220): Stopping 2023-07-11 14:36:11,295 INFO [RS:1;jenkins-hbase9:42359] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-11 14:36:11,295 INFO [RS:1;jenkins-hbase9:42359] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-11 14:36:11,295 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-11 14:36:11,295 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(3307): Received CLOSE for the region: 11796b6d2ba24476446363b9a5701d72, which we are already trying to CLOSE, but not completed yet 2023-07-11 14:36:11,295 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(3307): Received CLOSE for the region: 3bfd16063a0672430ddd816d03e73f73, which we are already trying to CLOSE, but not completed yet 2023-07-11 14:36:11,295 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(3307): Received CLOSE for the region: ef703c6f28122a7c70d67fbbd278729c, which we are already trying to CLOSE, but not completed yet 2023-07-11 14:36:11,295 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:11,295 DEBUG [RS:1;jenkins-hbase9:42359] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x71ca46e5 to 127.0.0.1:64930 2023-07-11 14:36:11,295 DEBUG [RS:1;jenkins-hbase9:42359] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:11,295 INFO [RS:1;jenkins-hbase9:42359] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-11 14:36:11,297 INFO [RS:1;jenkins-hbase9:42359] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-11 14:36:11,298 INFO [RS:1;jenkins-hbase9:42359] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-11 14:36:11,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 11796b6d2ba24476446363b9a5701d72, disabling compactions & flushes 2023-07-11 14:36:11,298 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:11,305 INFO [RS:0;jenkins-hbase9:37647] regionserver.HeapMemoryManager(220): Stopping 2023-07-11 14:36:11,302 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-11 14:36:11,306 INFO [RS:0;jenkins-hbase9:37647] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-11 14:36:11,302 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:11,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:11,307 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:11,307 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. after waiting 0 ms 2023-07-11 14:36:11,307 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:11,307 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 11796b6d2ba24476446363b9a5701d72 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-11 14:36:11,299 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-11 14:36:11,307 INFO [RS:0;jenkins-hbase9:37647] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-11 14:36:11,308 INFO [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:11,308 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-11 14:36:11,308 DEBUG [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1478): Online Regions={11796b6d2ba24476446363b9a5701d72=hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72., 1588230740=hbase:meta,,1.1588230740, 3bfd16063a0672430ddd816d03e73f73=t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73., ef703c6f28122a7c70d67fbbd278729c=t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.} 2023-07-11 14:36:11,308 DEBUG [RS:0;jenkins-hbase9:37647] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x17330197 to 127.0.0.1:64930 2023-07-11 14:36:11,308 DEBUG [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1504): Waiting on 11796b6d2ba24476446363b9a5701d72, 1588230740, 3bfd16063a0672430ddd816d03e73f73, ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:11,308 DEBUG [RS:0;jenkins-hbase9:37647] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:11,309 INFO [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,37647,1689086159883; all regions closed. 2023-07-11 14:36:11,310 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-11 14:36:11,310 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-11 14:36:11,310 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-11 14:36:11,310 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-11 14:36:11,310 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-11 14:36:11,311 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-07-11 14:36:11,318 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28917 2023-07-11 14:36:11,318 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-11 14:36:11,318 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28917 2023-07-11 14:36:11,318 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/ef703c6f28122a7c70d67fbbd278729c/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154, lastWalPosition=28917, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-07-11 14:36:11,318 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:11,321 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-11 14:36:11,321 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-11 14:36:11,321 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-11 14:36:11,321 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/3bfd16063a0672430ddd816d03e73f73/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237, lastWalPosition=14321, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-07-11 14:36:11,343 DEBUG [RS:0;jenkins-hbase9:37647] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs 2023-07-11 14:36:11,344 INFO [RS:0;jenkins-hbase9:37647] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0:(num 1689086161243) 2023-07-11 14:36:11,344 DEBUG [RS:0;jenkins-hbase9:37647] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:11,344 INFO [RS:0;jenkins-hbase9:37647] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:11,350 INFO [RS:0;jenkins-hbase9:37647] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-11 14:36:11,354 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/.tmp/info/aca6c0b18e134ac29b816a82dc14ccf7 2023-07-11 14:36:11,354 INFO [RS:0;jenkins-hbase9:37647] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-11 14:36:11,354 INFO [RS:0;jenkins-hbase9:37647] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-11 14:36:11,354 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:11,354 INFO [RS:0;jenkins-hbase9:37647] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-11 14:36:11,354 INFO [RS:0;jenkins-hbase9:37647] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-11 14:36:11,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/.tmp/info/d80a718065494749a24f9a2f7bd919d2 2023-07-11 14:36:11,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/.tmp/info/d80a718065494749a24f9a2f7bd919d2 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/info/d80a718065494749a24f9a2f7bd919d2 2023-07-11 14:36:11,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/info/d80a718065494749a24f9a2f7bd919d2, entries=2, sequenceid=6, filesize=4.8 K 2023-07-11 14:36:11,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 11796b6d2ba24476446363b9a5701d72 in 67ms, sequenceid=6, compaction requested=false 2023-07-11 14:36:11,374 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/.tmp/rep_barrier/1d3276f9ba23404ab9348dc7a8b8091a 2023-07-11 14:36:11,381 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-11 14:36:11,382 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:11,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 11796b6d2ba24476446363b9a5701d72: 2023-07-11 14:36:11,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:11,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 3bfd16063a0672430ddd816d03e73f73, disabling compactions & flushes 2023-07-11 14:36:11,383 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:11,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:11,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. after waiting 0 ms 2023-07-11 14:36:11,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:11,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 3bfd16063a0672430ddd816d03e73f73 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-07-11 14:36:11,394 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/.tmp/table/e62ac9c81e8e434e8393ec6d5abd21c0 2023-07-11 14:36:11,396 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,37647,1689086159883/jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0.1689086161243 was moved to hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs/jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0.1689086161243 2023-07-11 14:36:11,397 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/.tmp/cf1/08392405cee649fd8d2ff517b975a7cd 2023-07-11 14:36:11,402 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/.tmp/info/aca6c0b18e134ac29b816a82dc14ccf7 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/info/aca6c0b18e134ac29b816a82dc14ccf7 2023-07-11 14:36:11,404 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08392405cee649fd8d2ff517b975a7cd 2023-07-11 14:36:11,410 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/info/aca6c0b18e134ac29b816a82dc14ccf7, entries=30, sequenceid=19, filesize=8.1 K 2023-07-11 14:36:11,411 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/.tmp/rep_barrier/1d3276f9ba23404ab9348dc7a8b8091a as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/rep_barrier/1d3276f9ba23404ab9348dc7a8b8091a 2023-07-11 14:36:11,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/.tmp/norep/68eebf33a44e4b408009b187181e030d 2023-07-11 14:36:11,419 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/rep_barrier/1d3276f9ba23404ab9348dc7a8b8091a, entries=2, sequenceid=19, filesize=4.9 K 2023-07-11 14:36:11,421 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/.tmp/table/e62ac9c81e8e434e8393ec6d5abd21c0 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/table/e62ac9c81e8e434e8393ec6d5abd21c0 2023-07-11 14:36:11,424 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29747 2023-07-11 14:36:11,424 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-11 14:36:11,424 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29747 2023-07-11 14:36:11,426 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68eebf33a44e4b408009b187181e030d 2023-07-11 14:36:11,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/.tmp/cf1/08392405cee649fd8d2ff517b975a7cd as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/cf1/08392405cee649fd8d2ff517b975a7cd 2023-07-11 14:36:11,427 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-11 14:36:11,427 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-11 14:36:11,428 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-11 14:36:11,429 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/table/e62ac9c81e8e434e8393ec6d5abd21c0, entries=6, sequenceid=19, filesize=4.8 K 2023-07-11 14:36:11,430 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 120ms, sequenceid=19, compaction requested=false 2023-07-11 14:36:11,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08392405cee649fd8d2ff517b975a7cd 2023-07-11 14:36:11,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/cf1/08392405cee649fd8d2ff517b975a7cd, entries=100, sequenceid=106, filesize=8.2 K 2023-07-11 14:36:11,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/.tmp/norep/68eebf33a44e4b408009b187181e030d as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/norep/68eebf33a44e4b408009b187181e030d 2023-07-11 14:36:11,445 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-11 14:36:11,445 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-11 14:36:11,446 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-11 14:36:11,446 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-11 14:36:11,446 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-11 14:36:11,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68eebf33a44e4b408009b187181e030d 2023-07-11 14:36:11,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/norep/68eebf33a44e4b408009b187181e030d, entries=51, sequenceid=106, filesize=6.5 K 2023-07-11 14:36:11,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for 3bfd16063a0672430ddd816d03e73f73 in 70ms, sequenceid=106, compaction requested=false 2023-07-11 14:36:11,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-07-11 14:36:11,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:11,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 3bfd16063a0672430ddd816d03e73f73: 2023-07-11 14:36:11,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:11,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing ef703c6f28122a7c70d67fbbd278729c, disabling compactions & flushes 2023-07-11 14:36:11,466 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:11,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:11,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. after waiting 0 ms 2023-07-11 14:36:11,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:11,466 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing ef703c6f28122a7c70d67fbbd278729c 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-07-11 14:36:11,485 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/.tmp/cf1/a074c76829194557b0b7256c8de40f99 2023-07-11 14:36:11,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a074c76829194557b0b7256c8de40f99 2023-07-11 14:36:11,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/.tmp/norep/698a73b014474ad6a0a2f76f3101447c 2023-07-11 14:36:11,508 DEBUG [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1504): Waiting on ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:11,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 698a73b014474ad6a0a2f76f3101447c 2023-07-11 14:36:11,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/.tmp/cf1/a074c76829194557b0b7256c8de40f99 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/cf1/a074c76829194557b0b7256c8de40f99 2023-07-11 14:36:11,520 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a074c76829194557b0b7256c8de40f99 2023-07-11 14:36:11,520 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/cf1/a074c76829194557b0b7256c8de40f99, entries=200, sequenceid=206, filesize=11.9 K 2023-07-11 14:36:11,521 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/.tmp/norep/698a73b014474ad6a0a2f76f3101447c as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/norep/698a73b014474ad6a0a2f76f3101447c 2023-07-11 14:36:11,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 698a73b014474ad6a0a2f76f3101447c 2023-07-11 14:36:11,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/norep/698a73b014474ad6a0a2f76f3101447c, entries=101, sequenceid=206, filesize=8.3 K 2023-07-11 14:36:11,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for ef703c6f28122a7c70d67fbbd278729c in 63ms, sequenceid=206, compaction requested=false 2023-07-11 14:36:11,535 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 30301 2023-07-11 14:36:11,536 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-11 14:36:11,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-07-11 14:36:11,540 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:11,540 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for ef703c6f28122a7c70d67fbbd278729c: 2023-07-11 14:36:11,540 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:11,540 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-07-11 14:36:11,540 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-11 14:36:11,707 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b790017, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:11,707 DEBUG [RS:0;jenkins-hbase9:37647] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3a9a42b6 to 127.0.0.1:64930 2023-07-11 14:36:11,710 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,42359,1689086159924; all regions closed. 2023-07-11 14:36:11,710 DEBUG [RS:0;jenkins-hbase9:37647] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:11,711 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:11,718 DEBUG [RS:1;jenkins-hbase9:42359] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs 2023-07-11 14:36:11,718 INFO [RS:1;jenkins-hbase9:42359] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C42359%2C1689086159924.meta:.meta(num 1689086161410) 2023-07-11 14:36:11,718 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:11,724 DEBUG [RS:1;jenkins-hbase9:42359] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs 2023-07-11 14:36:11,724 INFO [RS:1;jenkins-hbase9:42359] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0:(num 1689086161237) 2023-07-11 14:36:11,724 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:11,729 DEBUG [RS:1;jenkins-hbase9:42359] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs 2023-07-11 14:36:11,729 INFO [RS:1;jenkins-hbase9:42359] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1:(num 1689086162154) 2023-07-11 14:36:11,729 DEBUG [RS:1;jenkins-hbase9:42359] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:11,729 INFO [RS:1;jenkins-hbase9:42359] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:11,730 INFO [RS:1;jenkins-hbase9:42359] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-11 14:36:11,730 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:11,730 INFO [RS:1;jenkins-hbase9:42359] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-11 14:36:11,811 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:381) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) 2023-07-11 14:36:11,812 INFO [RS:0;jenkins-hbase9:37647] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1 terminated 2023-07-11 14:36:11,813 INFO [RS:0;jenkins-hbase9:37647] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:37647 2023-07-11 14:36:11,817 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-11 14:36:11,817 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:11,817 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:11,817 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-11 14:36:11,817 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-11 14:36:11,817 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,37647,1689086159883] 2023-07-11 14:36:11,818 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,37647,1689086159883; numProcessing=1 2023-07-11 14:36:11,822 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase9.apache.org,37647,1689086159883 already deleted, retry=false 2023-07-11 14:36:11,822 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,37647,1689086159883 expired; onlineServers=1 2023-07-11 14:36:11,912 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b790018, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:11,912 DEBUG [RS:1;jenkins-hbase9:42359] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x078af769 to 127.0.0.1:64930 2023-07-11 14:36:11,912 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b790018, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:11,912 DEBUG [RS:1;jenkins-hbase9:42359] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:11,953 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-11 14:36:12,012 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:12,013 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:37647-0x10154c21b790001, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:12,013 INFO [RS:0;jenkins-hbase9:37647] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,37647,1689086159883; zookeeper connection closed. 2023-07-11 14:36:12,015 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@34b39cac] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@34b39cac 2023-07-11 14:36:12,113 INFO [RS:1;jenkins-hbase9:42359] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1 terminated 2023-07-11 14:36:12,114 INFO [RS:1;jenkins-hbase9:42359] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase9:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1 terminated 2023-07-11 14:36:12,115 INFO [RS:1;jenkins-hbase9:42359] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:42359 2023-07-11 14:36:12,118 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:12,119 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-11 14:36:12,119 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,42359,1689086159924] 2023-07-11 14:36:12,119 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,42359,1689086159924; numProcessing=2 2023-07-11 14:36:12,123 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase9.apache.org,42359,1689086159924 already deleted, retry=false 2023-07-11 14:36:12,123 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,42359,1689086159924 expired; onlineServers=0 2023-07-11 14:36:12,123 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,46131,1689086158709' ***** 2023-07-11 14:36:12,123 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-11 14:36:12,124 DEBUG [M:0;jenkins-hbase9:46131] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@f89f12f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:12,124 INFO [M:0;jenkins-hbase9:46131] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:36:12,124 INFO [M:0;jenkins-hbase9:46131] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,46131,1689086158709; all regions closed. 2023-07-11 14:36:12,124 DEBUG [M:0;jenkins-hbase9:46131] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:12,124 DEBUG [M:0;jenkins-hbase9:46131] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-11 14:36:12,124 DEBUG [M:0;jenkins-hbase9:46131] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-11 14:36:12,124 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086160900] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086160900,5,FailOnTimeoutGroup] 2023-07-11 14:36:12,125 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-11 14:36:12,125 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086160899] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086160899,5,FailOnTimeoutGroup] 2023-07-11 14:36:12,124 INFO [M:0;jenkins-hbase9:46131] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-11 14:36:12,125 INFO [M:0;jenkins-hbase9:46131] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-11 14:36:12,125 INFO [M:0;jenkins-hbase9:46131] hbase.ChoreService(369): Chore service for: master/jenkins-hbase9:0 had [] on shutdown 2023-07-11 14:36:12,125 DEBUG [M:0;jenkins-hbase9:46131] master.HMaster(1512): Stopping service threads 2023-07-11 14:36:12,125 INFO [M:0;jenkins-hbase9:46131] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-11 14:36:12,125 ERROR [M:0;jenkins-hbase9:46131] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-07-11 14:36:12,126 INFO [M:0;jenkins-hbase9:46131] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-11 14:36:12,126 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-11 14:36:12,126 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-11 14:36:12,126 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:12,126 DEBUG [M:0;jenkins-hbase9:46131] zookeeper.ZKUtil(398): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-11 14:36:12,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-11 14:36:12,126 WARN [M:0;jenkins-hbase9:46131] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-11 14:36:12,126 INFO [M:0;jenkins-hbase9:46131] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-11 14:36:12,127 INFO [M:0;jenkins-hbase9:46131] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-11 14:36:12,128 DEBUG [M:0;jenkins-hbase9:46131] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-11 14:36:12,128 INFO [M:0;jenkins-hbase9:46131] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:12,128 DEBUG [M:0;jenkins-hbase9:46131] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:12,128 DEBUG [M:0;jenkins-hbase9:46131] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-11 14:36:12,128 DEBUG [M:0;jenkins-hbase9:46131] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:12,128 INFO [M:0;jenkins-hbase9:46131] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.04 KB heapSize=72.60 KB 2023-07-11 14:36:12,139 INFO [M:0;jenkins-hbase9:46131] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.04 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d2f4bca8d89c4d48af6329fae0cffa15 2023-07-11 14:36:12,145 DEBUG [M:0;jenkins-hbase9:46131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d2f4bca8d89c4d48af6329fae0cffa15 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d2f4bca8d89c4d48af6329fae0cffa15 2023-07-11 14:36:12,150 INFO [M:0;jenkins-hbase9:46131] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d2f4bca8d89c4d48af6329fae0cffa15, entries=17, sequenceid=136, filesize=8.5 K 2023-07-11 14:36:12,151 INFO [M:0;jenkins-hbase9:46131] regionserver.HRegion(2948): Finished flush of dataSize ~61.04 KB/62508, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=136, compaction requested=false 2023-07-11 14:36:12,153 INFO [M:0;jenkins-hbase9:46131] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:12,153 DEBUG [M:0;jenkins-hbase9:46131] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:12,153 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,46131,1689086158709 2023-07-11 14:36:12,161 INFO [M:0;jenkins-hbase9:46131] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-11 14:36:12,161 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:12,162 INFO [M:0;jenkins-hbase9:46131] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:46131 2023-07-11 14:36:12,164 DEBUG [M:0;jenkins-hbase9:46131] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase9.apache.org,46131,1689086158709 already deleted, retry=false 2023-07-11 14:36:12,222 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:12,222 INFO [RS:1;jenkins-hbase9:42359] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,42359,1689086159924; zookeeper connection closed. 2023-07-11 14:36:12,222 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): regionserver:42359-0x10154c21b790002, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:12,222 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@724e9b95] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@724e9b95 2023-07-11 14:36:12,222 INFO [Listener at localhost/42861] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-07-11 14:36:12,322 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:12,322 INFO [M:0;jenkins-hbase9:46131] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,46131,1689086158709; zookeeper connection closed. 2023-07-11 14:36:12,322 DEBUG [Listener at localhost/36371-EventThread] zookeeper.ZKWatcher(600): master:46131-0x10154c21b790000, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:12,325 INFO [Listener at localhost/42861] client.ConnectionUtils(127): master/jenkins-hbase9:0 server-side Connection retries=18 2023-07-11 14:36:12,325 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:12,325 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:12,325 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:12,325 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:12,325 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:12,325 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:12,327 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:32785 2023-07-11 14:36:12,328 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:12,329 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:12,330 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=master:32785 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:12,333 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:327850x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:12,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:32785-0x10154c21b79001c connected 2023-07-11 14:36:12,336 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:12,337 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:12,337 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-11 14:36:12,337 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=32785 2023-07-11 14:36:12,341 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=32785 2023-07-11 14:36:12,342 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=32785 2023-07-11 14:36:12,342 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=32785 2023-07-11 14:36:12,342 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=32785 2023-07-11 14:36:12,343 INFO [Listener at localhost/42861] master.HMaster(444): hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902, hbase.cluster.distributed=false 2023-07-11 14:36:12,343 DEBUG [pool-452-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-11 14:36:12,361 INFO [Listener at localhost/42861] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=18 2023-07-11 14:36:12,361 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:12,362 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:12,362 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:12,362 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:12,362 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:12,362 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:12,365 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:42891 2023-07-11 14:36:12,365 INFO [Listener at localhost/42861] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-11 14:36:12,366 DEBUG [Listener at localhost/42861] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-11 14:36:12,367 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:12,368 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:12,369 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42891 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:12,380 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:428910x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:12,382 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:428910x0, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:12,383 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:428910x0, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:12,383 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42891-0x10154c21b79001d connected 2023-07-11 14:36:12,383 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-11 14:36:12,384 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42891 2023-07-11 14:36:12,385 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42891 2023-07-11 14:36:12,386 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42891 2023-07-11 14:36:12,386 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42891 2023-07-11 14:36:12,386 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42891 2023-07-11 14:36:12,397 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,399 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-11 14:36:12,400 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,402 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-11 14:36:12,402 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-11 14:36:12,403 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:12,403 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-11 14:36:12,405 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase9.apache.org,32785,1689086172324 from backup master directory 2023-07-11 14:36:12,405 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-11 14:36:12,406 WARN [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:12,407 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,407 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,407 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-11 14:36:12,423 INFO [master/jenkins-hbase9:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:12,454 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x433cbd50 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:12,465 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@60ed745c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:12,465 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:12,466 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-11 14:36:12,466 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:12,467 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:12,478 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,34835,1689086163812 to hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,34835,1689086163812-dead as it is dead 2023-07-11 14:36:12,480 INFO [master/jenkins-hbase9:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,34835,1689086163812-dead/jenkins-hbase9.apache.org%2C34835%2C1689086163812.jenkins-hbase9.apache.org%2C34835%2C1689086163812.regiongroup-0.1689086164091 2023-07-11 14:36:12,486 INFO [master/jenkins-hbase9:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,34835,1689086163812-dead/jenkins-hbase9.apache.org%2C34835%2C1689086163812.jenkins-hbase9.apache.org%2C34835%2C1689086163812.regiongroup-0.1689086164091 after 6ms 2023-07-11 14:36:12,487 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,34835,1689086163812-dead/jenkins-hbase9.apache.org%2C34835%2C1689086163812.jenkins-hbase9.apache.org%2C34835%2C1689086163812.regiongroup-0.1689086164091 to hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C34835%2C1689086163812.jenkins-hbase9.apache.org%2C34835%2C1689086163812.regiongroup-0.1689086164091 2023-07-11 14:36:12,487 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,34835,1689086163812-dead 2023-07-11 14:36:12,488 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,489 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:12,490 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C32785%2C1689086172324.jenkins-hbase9.apache.org%2C32785%2C1689086172324.regiongroup-0, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,32785,1689086172324, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/oldWALs, maxLogs=10 2023-07-11 14:36:12,521 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,32785,1689086172324/jenkins-hbase9.apache.org%2C32785%2C1689086172324.jenkins-hbase9.apache.org%2C32785%2C1689086172324.regiongroup-0.1689086172491 2023-07-11 14:36:12,521 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK], DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK], DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK]] 2023-07-11 14:36:12,522 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:12,522 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:12,522 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:12,522 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:12,525 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:12,526 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-11 14:36:12,527 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-11 14:36:12,539 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bafe90e861e748fda9018cd963db289a 2023-07-11 14:36:12,540 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:12,541 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-11 14:36:12,541 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C34835%2C1689086163812.jenkins-hbase9.apache.org%2C34835%2C1689086163812.regiongroup-0.1689086164091 2023-07-11 14:36:12,548 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C34835%2C1689086163812.jenkins-hbase9.apache.org%2C34835%2C1689086163812.regiongroup-0.1689086164091 2023-07-11 14:36:12,550 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C34835%2C1689086163812.jenkins-hbase9.apache.org%2C34835%2C1689086163812.regiongroup-0.1689086164091 2023-07-11 14:36:12,554 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:12,560 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-07-11 14:36:12,562 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67003853, jitterRate=-0.0015647858381271362}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-11 14:36:12,562 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:12,563 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-11 14:36:12,565 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-11 14:36:12,565 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-11 14:36:12,565 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-11 14:36:12,566 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-11 14:36:12,583 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-11 14:36:12,583 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-11 14:36:12,584 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-11 14:36:12,584 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-11 14:36:12,584 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-11 14:36:12,584 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-11 14:36:12,585 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 18 msec 2023-07-11 14:36:12,585 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-11 14:36:12,585 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-11 14:36:12,586 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,41517,1689086163885, table=hbase:meta, region=1588230740 2023-07-11 14:36:12,588 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-11 14:36:12,590 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,37977,1689086163856 already deleted, retry=false 2023-07-11 14:36:12,590 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase9.apache.org,37977,1689086163856 on jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,600 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase9.apache.org,37977,1689086163856, splitWal=true, meta=false 2023-07-11 14:36:12,600 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase9.apache.org,37977,1689086163856 (carryingMeta=false) jenkins-hbase9.apache.org,37977,1689086163856/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@40d53bd9[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-11 14:36:12,603 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,41517,1689086163885 already deleted, retry=false 2023-07-11 14:36:12,603 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase9.apache.org,41517,1689086163885 on jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,604 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase9.apache.org,41517,1689086163885, splitWal=true, meta=true 2023-07-11 14:36:12,605 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase9.apache.org,41517,1689086163885 (carryingMeta=true) jenkins-hbase9.apache.org,41517,1689086163885/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@70f9eacd[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-11 14:36:12,606 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,42167,1689086163940 already deleted, retry=false 2023-07-11 14:36:12,606 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase9.apache.org,42167,1689086163940 on jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,607 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase9.apache.org,42167,1689086163940, splitWal=true, meta=false 2023-07-11 14:36:12,607 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase9.apache.org,42167,1689086163940 (carryingMeta=false) jenkins-hbase9.apache.org,42167,1689086163940/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@191b1cf8[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-11 14:36:12,609 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,34239,1689086163911 already deleted, retry=false 2023-07-11 14:36:12,609 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase9.apache.org,34239,1689086163911 on jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,610 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase9.apache.org,34239,1689086163911, splitWal=true, meta=false 2023-07-11 14:36:12,610 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase9.apache.org,34239,1689086163911 (carryingMeta=false) jenkins-hbase9.apache.org,34239,1689086163911/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@6fd9d732[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-11 14:36:12,624 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-11 14:36:12,624 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-11 14:36:12,624 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-11 14:36:12,624 INFO [master/jenkins-hbase9:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-11 14:36:12,625 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-11 14:36:12,626 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-11 14:36:12,626 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-11 14:36:12,627 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-11 14:36:12,628 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-11 14:36:12,628 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-11 14:36:12,629 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:12,629 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase9.apache.org,32785,1689086172324, sessionid=0x10154c21b79001c, setting cluster-up flag (Was=false) 2023-07-11 14:36:12,635 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-11 14:36:12,637 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,640 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-11 14:36:12,642 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:12,642 WARN [master/jenkins-hbase9:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.hbase-snapshot/.tmp 2023-07-11 14:36:12,646 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:12,646 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:12,646 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:12,646 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:12,646 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase9:0, corePoolSize=10, maxPoolSize=10 2023-07-11 14:36:12,646 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,646 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:12,646 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,647 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689086202647 2023-07-11 14:36:12,647 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-11 14:36:12,647 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-11 14:36:12,647 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-11 14:36:12,647 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-11 14:36:12,647 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-11 14:36:12,648 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-11 14:36:12,650 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:12,650 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-11 14:36:12,650 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-11 14:36:12,650 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-11 14:36:12,651 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-11 14:36:12,651 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-11 14:36:12,652 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086172651,5,FailOnTimeoutGroup] 2023-07-11 14:36:12,652 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086172652,5,FailOnTimeoutGroup] 2023-07-11 14:36:12,653 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:12,653 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-11 14:36:12,653 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:12,653 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:12,653 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689086172653, completionTime=-1 2023-07-11 14:36:12,653 WARN [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-11 14:36:12,653 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-11 14:36:12,655 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase9.apache.org,37977,1689086163856; numProcessing=1 2023-07-11 14:36:12,655 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase9.apache.org,42167,1689086163940; numProcessing=2 2023-07-11 14:36:12,655 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase9.apache.org,41517,1689086163885; numProcessing=3 2023-07-11 14:36:12,655 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase9.apache.org,34239,1689086163911; numProcessing=4 2023-07-11 14:36:12,656 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,37977,1689086163856, splitWal=true, meta=false 2023-07-11 14:36:12,656 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,42167,1689086163940, splitWal=true, meta=false 2023-07-11 14:36:12,656 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,34239,1689086163911, splitWal=true, meta=false 2023-07-11 14:36:12,656 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,41517,1689086163885, splitWal=true, meta=true 2023-07-11 14:36:12,663 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,41517,1689086163885, splitWal=true, meta=true, isMeta: true 2023-07-11 14:36:12,668 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885-splitting 2023-07-11 14:36:12,669 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885-splitting dir is empty, no logs to split. 2023-07-11 14:36:12,669 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase9.apache.org,41517,1689086163885 WAL count=0, meta=true 2023-07-11 14:36:12,673 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885-splitting dir is empty, no logs to split. 2023-07-11 14:36:12,673 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase9.apache.org,41517,1689086163885 WAL count=0, meta=true 2023-07-11 14:36:12,674 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase9.apache.org,41517,1689086163885 WAL splitting is done? wals=0, meta=true 2023-07-11 14:36:12,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-11 14:36:12,677 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-11 14:36:12,679 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:12,695 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(951): ClusterId : 6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:12,695 DEBUG [RS:0;jenkins-hbase9:42891] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-11 14:36:12,697 DEBUG [RS:0;jenkins-hbase9:42891] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-11 14:36:12,697 DEBUG [RS:0;jenkins-hbase9:42891] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-11 14:36:12,701 DEBUG [RS:0;jenkins-hbase9:42891] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-11 14:36:12,703 DEBUG [RS:0;jenkins-hbase9:42891] zookeeper.ReadOnlyZKClient(139): Connect 0x27c81d4b to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:12,714 DEBUG [RS:0;jenkins-hbase9:42891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@290fe7c2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:12,715 DEBUG [RS:0;jenkins-hbase9:42891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d30763b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:12,725 DEBUG [RS:0;jenkins-hbase9:42891] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase9:42891 2023-07-11 14:36:12,725 INFO [RS:0;jenkins-hbase9:42891] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-11 14:36:12,725 INFO [RS:0;jenkins-hbase9:42891] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-11 14:36:12,725 DEBUG [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1022): About to register with Master. 2023-07-11 14:36:12,726 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase9.apache.org,32785,1689086172324 with isa=jenkins-hbase9.apache.org/172.31.2.10:42891, startcode=1689086172361 2023-07-11 14:36:12,726 DEBUG [RS:0;jenkins-hbase9:42891] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-11 14:36:12,730 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:40313, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-11 14:36:12,731 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=32785] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:12,731 DEBUG [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902 2023-07-11 14:36:12,731 DEBUG [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33985 2023-07-11 14:36:12,731 DEBUG [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-11 14:36:12,733 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:12,734 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,42891,1689086172361] 2023-07-11 14:36:12,734 DEBUG [RS:0;jenkins-hbase9:42891] zookeeper.ZKUtil(162): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:12,734 WARN [RS:0;jenkins-hbase9:42891] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:12,735 INFO [RS:0;jenkins-hbase9:42891] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:12,735 INFO [RS:0;jenkins-hbase9:42891] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:12,735 DEBUG [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:12,738 DEBUG [RS:0;jenkins-hbase9:42891] zookeeper.ZKUtil(162): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:12,739 DEBUG [RS:0;jenkins-hbase9:42891] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:12,739 INFO [RS:0;jenkins-hbase9:42891] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-11 14:36:12,750 INFO [RS:0;jenkins-hbase9:42891] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-11 14:36:12,750 INFO [RS:0;jenkins-hbase9:42891] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-11 14:36:12,750 INFO [RS:0;jenkins-hbase9:42891] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:12,753 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-11 14:36:12,753 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=100ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-11 14:36:12,754 INFO [RS:0;jenkins-hbase9:42891] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:12,754 DEBUG [RS:0;jenkins-hbase9:42891] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,754 DEBUG [RS:0;jenkins-hbase9:42891] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,754 DEBUG [RS:0;jenkins-hbase9:42891] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,754 DEBUG [RS:0;jenkins-hbase9:42891] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,754 DEBUG [RS:0;jenkins-hbase9:42891] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,754 DEBUG [RS:0;jenkins-hbase9:42891] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:12,755 DEBUG [RS:0;jenkins-hbase9:42891] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,755 DEBUG [RS:0;jenkins-hbase9:42891] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,755 DEBUG [RS:0;jenkins-hbase9:42891] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,755 DEBUG [RS:0;jenkins-hbase9:42891] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:12,757 INFO [RS:0;jenkins-hbase9:42891] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:12,757 INFO [RS:0;jenkins-hbase9:42891] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:12,758 INFO [RS:0;jenkins-hbase9:42891] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:12,770 INFO [RS:0;jenkins-hbase9:42891] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-11 14:36:12,770 INFO [RS:0;jenkins-hbase9:42891] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,42891,1689086172361-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:12,792 INFO [RS:0;jenkins-hbase9:42891] regionserver.Replication(203): jenkins-hbase9.apache.org,42891,1689086172361 started 2023-07-11 14:36:12,792 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1637): Serving as jenkins-hbase9.apache.org,42891,1689086172361, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:42891, sessionid=0x10154c21b79001d 2023-07-11 14:36:12,792 DEBUG [RS:0;jenkins-hbase9:42891] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-11 14:36:12,792 DEBUG [RS:0;jenkins-hbase9:42891] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:12,792 DEBUG [RS:0;jenkins-hbase9:42891] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,42891,1689086172361' 2023-07-11 14:36:12,793 DEBUG [RS:0;jenkins-hbase9:42891] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-11 14:36:12,793 DEBUG [RS:0;jenkins-hbase9:42891] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-11 14:36:12,794 DEBUG [RS:0;jenkins-hbase9:42891] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-11 14:36:12,794 DEBUG [RS:0;jenkins-hbase9:42891] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-11 14:36:12,794 DEBUG [RS:0;jenkins-hbase9:42891] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:12,794 DEBUG [RS:0;jenkins-hbase9:42891] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,42891,1689086172361' 2023-07-11 14:36:12,794 DEBUG [RS:0;jenkins-hbase9:42891] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-11 14:36:12,794 DEBUG [RS:0;jenkins-hbase9:42891] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-11 14:36:12,794 DEBUG [RS:0;jenkins-hbase9:42891] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-11 14:36:12,795 INFO [RS:0;jenkins-hbase9:42891] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-11 14:36:12,795 INFO [RS:0;jenkins-hbase9:42891] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-11 14:36:12,830 DEBUG [jenkins-hbase9:32785] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-11 14:36:12,831 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,42891,1689086172361, state=OPENING 2023-07-11 14:36:12,833 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-11 14:36:12,834 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-11 14:36:12,834 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,42891,1689086172361}] 2023-07-11 14:36:12,895 INFO [RS:0;jenkins-hbase9:42891] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:12,897 INFO [RS:0;jenkins-hbase9:42891] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C42891%2C1689086172361.jenkins-hbase9.apache.org%2C42891%2C1689086172361.regiongroup-0, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:12,908 INFO [RS:0;jenkins-hbase9:42891] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361/jenkins-hbase9.apache.org%2C42891%2C1689086172361.jenkins-hbase9.apache.org%2C42891%2C1689086172361.regiongroup-0.1689086172898 2023-07-11 14:36:12,908 DEBUG [RS:0;jenkins-hbase9:42891] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK], DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK], DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK]] 2023-07-11 14:36:12,988 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:12,989 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:12,991 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:33414, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-11 14:36:12,995 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-11 14:36:12,995 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:12,995 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:12,995 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:12,997 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C42891%2C1689086172361.meta, suffix=.meta, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:13,010 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361/jenkins-hbase9.apache.org%2C42891%2C1689086172361.meta.1689086172997.meta 2023-07-11 14:36:13,010 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK], DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK], DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK]] 2023-07-11 14:36:13,010 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:13,010 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-11 14:36:13,010 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-11 14:36:13,011 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-11 14:36:13,011 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-11 14:36:13,011 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:13,011 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-11 14:36:13,012 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-11 14:36:13,014 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-11 14:36:13,015 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info 2023-07-11 14:36:13,015 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info 2023-07-11 14:36:13,015 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-11 14:36:13,021 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info/2023d570cfaa402593275b0451eb9042 2023-07-11 14:36:13,021 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:13,022 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-11 14:36:13,022 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:13,023 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:13,023 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-11 14:36:13,024 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:13,024 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-11 14:36:13,025 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table 2023-07-11 14:36:13,025 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table 2023-07-11 14:36:13,025 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-11 14:36:13,031 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table/ed3b6a6f597941748150410cb9e7546c 2023-07-11 14:36:13,031 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:13,032 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740 2023-07-11 14:36:13,033 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740 2023-07-11 14:36:13,036 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-11 14:36:13,037 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-11 14:36:13,038 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65790389, jitterRate=-0.01964680850505829}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-11 14:36:13,038 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-11 14:36:13,041 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1689086172988 2023-07-11 14:36:13,045 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-11 14:36:13,045 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-11 14:36:13,046 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,42891,1689086172361, state=OPEN 2023-07-11 14:36:13,048 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-11 14:36:13,049 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-11 14:36:13,051 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-11 14:36:13,052 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,42891,1689086172361 in 215 msec 2023-07-11 14:36:13,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=16 2023-07-11 14:36:13,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 377 msec 2023-07-11 14:36:14,256 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1603ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-11 14:36:15,758 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3105ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-11 14:36:16,885 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-11 14:36:17,161 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4508ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-11 14:36:17,161 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-11 14:36:17,165 DEBUG [hconnection-0x336806e4-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:17,167 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:33416, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:17,176 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=d4185bdf44467f073faaa5de49a197d7, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41517,1689086163885, regionLocation=jenkins-hbase9.apache.org,41517,1689086163885, openSeqNum=2 2023-07-11 14:36:17,177 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=f27301259d11533bab9412d981c5782d, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,42167,1689086163940, regionLocation=jenkins-hbase9.apache.org,42167,1689086163940, openSeqNum=2 2023-07-11 14:36:17,177 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=34f512aa6fa4c9880527ac501e69b8b2, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,42167,1689086163940, regionLocation=jenkins-hbase9.apache.org,42167,1689086163940, openSeqNum=2 2023-07-11 14:36:17,177 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-11 14:36:17,177 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689086237177 2023-07-11 14:36:17,177 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689086297177 2023-07-11 14:36:17,177 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 16 msec 2023-07-11 14:36:17,207 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,32785,1689086172324-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:17,207 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,32785,1689086172324-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:17,207 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,32785,1689086172324-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:17,207 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase9.apache.org,37977,1689086163856 had 0 regions 2023-07-11 14:36:17,207 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase9:32785, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:17,207 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase9.apache.org,34239,1689086163911 had 0 regions 2023-07-11 14:36:17,207 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase9.apache.org,41517,1689086163885 had 2 regions 2023-07-11 14:36:17,207 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase9.apache.org,42167,1689086163940 had 2 regions 2023-07-11 14:36:17,207 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:17,208 WARN [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. is NOT online; state={d4185bdf44467f073faaa5de49a197d7 state=OPEN, ts=1689086177176, server=jenkins-hbase9.apache.org,41517,1689086163885}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-11 14:36:17,208 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,34239,1689086163911, splitWal=true, meta=false, isMeta: false 2023-07-11 14:36:17,208 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,37977,1689086163856, splitWal=true, meta=false, isMeta: false 2023-07-11 14:36:17,209 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,41517,1689086163885, splitWal=true, meta=true, isMeta: false 2023-07-11 14:36:17,209 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,42167,1689086163940, splitWal=true, meta=false, isMeta: false 2023-07-11 14:36:17,210 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,34239,1689086163911-splitting 2023-07-11 14:36:17,212 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,34239,1689086163911-splitting dir is empty, no logs to split. 2023-07-11 14:36:17,212 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase9.apache.org,34239,1689086163911 WAL count=0, meta=false 2023-07-11 14:36:17,212 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,37977,1689086163856-splitting 2023-07-11 14:36:17,213 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,37977,1689086163856-splitting dir is empty, no logs to split. 2023-07-11 14:36:17,213 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase9.apache.org,37977,1689086163856 WAL count=0, meta=false 2023-07-11 14:36:17,214 WARN [master/jenkins-hbase9:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase9.apache.org,41517,1689086163885/hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7., unknown_server=jenkins-hbase9.apache.org,42167,1689086163940/t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d., unknown_server=jenkins-hbase9.apache.org,42167,1689086163940/t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:17,215 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885-splitting dir is empty, no logs to split. 2023-07-11 14:36:17,215 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase9.apache.org,41517,1689086163885 WAL count=0, meta=false 2023-07-11 14:36:17,215 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940-splitting 2023-07-11 14:36:17,216 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940-splitting dir is empty, no logs to split. 2023-07-11 14:36:17,216 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase9.apache.org,42167,1689086163940 WAL count=0, meta=false 2023-07-11 14:36:17,217 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,34239,1689086163911-splitting dir is empty, no logs to split. 2023-07-11 14:36:17,217 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase9.apache.org,34239,1689086163911 WAL count=0, meta=false 2023-07-11 14:36:17,217 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase9.apache.org,34239,1689086163911 WAL splitting is done? wals=0, meta=false 2023-07-11 14:36:17,218 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,37977,1689086163856-splitting dir is empty, no logs to split. 2023-07-11 14:36:17,218 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase9.apache.org,37977,1689086163856 WAL count=0, meta=false 2023-07-11 14:36:17,218 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase9.apache.org,37977,1689086163856 WAL splitting is done? wals=0, meta=false 2023-07-11 14:36:17,219 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,41517,1689086163885-splitting dir is empty, no logs to split. 2023-07-11 14:36:17,219 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase9.apache.org,41517,1689086163885 WAL count=0, meta=false 2023-07-11 14:36:17,219 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase9.apache.org,41517,1689086163885 WAL splitting is done? wals=0, meta=false 2023-07-11 14:36:17,219 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940-splitting dir is empty, no logs to split. 2023-07-11 14:36:17,220 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase9.apache.org,42167,1689086163940 WAL count=0, meta=false 2023-07-11 14:36:17,220 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase9.apache.org,42167,1689086163940 WAL splitting is done? wals=0, meta=false 2023-07-11 14:36:17,220 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase9.apache.org,34239,1689086163911 failed, ignore...File hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,34239,1689086163911-splitting does not exist. 2023-07-11 14:36:17,220 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase9.apache.org,37977,1689086163856 failed, ignore...File hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,37977,1689086163856-splitting does not exist. 2023-07-11 14:36:17,221 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN}] 2023-07-11 14:36:17,222 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase9.apache.org,34239,1689086163911 after splitting done 2023-07-11 14:36:17,222 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase9.apache.org,34239,1689086163911 from processing; numProcessing=3 2023-07-11 14:36:17,222 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase9.apache.org,37977,1689086163856 after splitting done 2023-07-11 14:36:17,222 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase9.apache.org,37977,1689086163856 from processing; numProcessing=2 2023-07-11 14:36:17,222 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN 2023-07-11 14:36:17,223 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase9.apache.org,42167,1689086163940 failed, ignore...File hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42167,1689086163940-splitting does not exist. 2023-07-11 14:36:17,224 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:17,224 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN}, {pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN}] 2023-07-11 14:36:17,225 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase9.apache.org,34239,1689086163911, splitWal=true, meta=false in 4.6130 sec 2023-07-11 14:36:17,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase9.apache.org,37977,1689086163856, splitWal=true, meta=false in 4.6300 sec 2023-07-11 14:36:17,226 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN 2023-07-11 14:36:17,226 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN 2023-07-11 14:36:17,227 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:17,227 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:17,374 DEBUG [jenkins-hbase9:32785] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-11 14:36:17,377 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=d4185bdf44467f073faaa5de49a197d7, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:17,377 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=34f512aa6fa4c9880527ac501e69b8b2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:17,377 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=f27301259d11533bab9412d981c5782d, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:17,377 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086177377"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086177377"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086177377"}]},"ts":"1689086177377"} 2023-07-11 14:36:17,377 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086177377"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086177377"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086177377"}]},"ts":"1689086177377"} 2023-07-11 14:36:17,377 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086177377"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086177377"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086177377"}]},"ts":"1689086177377"} 2023-07-11 14:36:17,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE; OpenRegionProcedure 34f512aa6fa4c9880527ac501e69b8b2, server=jenkins-hbase9.apache.org,42891,1689086172361}] 2023-07-11 14:36:17,382 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure f27301259d11533bab9412d981c5782d, server=jenkins-hbase9.apache.org,42891,1689086172361}] 2023-07-11 14:36:17,383 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=21, state=RUNNABLE; OpenRegionProcedure d4185bdf44467f073faaa5de49a197d7, server=jenkins-hbase9.apache.org,42891,1689086172361}] 2023-07-11 14:36:17,538 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:17,539 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:17,540 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C42891%2C1689086172361.jenkins-hbase9.apache.org%2C42891%2C1689086172361.regiongroup-1, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:17,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:17,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 34f512aa6fa4c9880527ac501e69b8b2, NAME => 't2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:17,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:17,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:17,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:17,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:17,543 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:17,544 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1 2023-07-11 14:36:17,544 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1 2023-07-11 14:36:17,545 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 34f512aa6fa4c9880527ac501e69b8b2 columnFamilyName cf1 2023-07-11 14:36:17,550 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361/jenkins-hbase9.apache.org%2C42891%2C1689086172361.jenkins-hbase9.apache.org%2C42891%2C1689086172361.regiongroup-1.1689086177541 2023-07-11 14:36:17,550 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK], DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK], DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK]] 2023-07-11 14:36:17,550 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d4185bdf44467f073faaa5de49a197d7, NAME => 'hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:17,550 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:17,550 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:17,551 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:17,551 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:17,552 INFO [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:17,554 DEBUG [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/info 2023-07-11 14:36:17,554 DEBUG [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/info 2023-07-11 14:36:17,554 INFO [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4185bdf44467f073faaa5de49a197d7 columnFamilyName info 2023-07-11 14:36:17,555 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1/e3e0f680c76e48bf80b60121dfd11e86 2023-07-11 14:36:17,555 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(310): Store=34f512aa6fa4c9880527ac501e69b8b2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:17,555 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:17,556 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/norep 2023-07-11 14:36:17,556 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/norep 2023-07-11 14:36:17,556 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 34f512aa6fa4c9880527ac501e69b8b2 columnFamilyName norep 2023-07-11 14:36:17,557 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(310): Store=34f512aa6fa4c9880527ac501e69b8b2/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:17,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:17,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:17,560 DEBUG [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/info/ca4f495a328442daad2ea4bf64ee841b 2023-07-11 14:36:17,560 INFO [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] regionserver.HStore(310): Store=d4185bdf44467f073faaa5de49a197d7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:17,561 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:17,562 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:17,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-11 14:36:17,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:17,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 34f512aa6fa4c9880527ac501e69b8b2; next sequenceid=12; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65605109, jitterRate=-0.022407695651054382}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-11 14:36:17,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 34f512aa6fa4c9880527ac501e69b8b2: 2023-07-11 14:36:17,568 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:17,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2., pid=24, masterSystemTime=1689086177534 2023-07-11 14:36:17,569 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened d4185bdf44467f073faaa5de49a197d7; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59642042, jitterRate=-0.11126431822776794}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-11 14:36:17,569 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for d4185bdf44467f073faaa5de49a197d7: 2023-07-11 14:36:17,573 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7., pid=26, masterSystemTime=1689086177534 2023-07-11 14:36:17,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:17,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:17,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:17,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f27301259d11533bab9412d981c5782d, NAME => 't1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:17,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup f27301259d11533bab9412d981c5782d 2023-07-11 14:36:17,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:17,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:17,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:17,575 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=34f512aa6fa4c9880527ac501e69b8b2, regionState=OPEN, openSeqNum=12, regionLocation=jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:17,576 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086177575"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086177575"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086177575"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086177575"}]},"ts":"1689086177575"} 2023-07-11 14:36:17,576 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:17,576 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:17,576 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f27301259d11533bab9412d981c5782d 2023-07-11 14:36:17,577 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=d4185bdf44467f073faaa5de49a197d7, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:17,577 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086177577"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086177577"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086177577"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086177577"}]},"ts":"1689086177577"} 2023-07-11 14:36:17,578 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1 2023-07-11 14:36:17,578 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1 2023-07-11 14:36:17,578 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f27301259d11533bab9412d981c5782d columnFamilyName cf1 2023-07-11 14:36:17,580 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-07-11 14:36:17,581 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; OpenRegionProcedure 34f512aa6fa4c9880527ac501e69b8b2, server=jenkins-hbase9.apache.org,42891,1689086172361 in 197 msec 2023-07-11 14:36:17,582 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=21 2023-07-11 14:36:17,582 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=21, state=SUCCESS; OpenRegionProcedure d4185bdf44467f073faaa5de49a197d7, server=jenkins-hbase9.apache.org,42891,1689086172361 in 196 msec 2023-07-11 14:36:17,583 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN in 357 msec 2023-07-11 14:36:17,584 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=16 2023-07-11 14:36:17,585 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase9.apache.org,41517,1689086163885 after splitting done 2023-07-11 14:36:17,585 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN in 361 msec 2023-07-11 14:36:17,585 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase9.apache.org,41517,1689086163885 from processing; numProcessing=1 2023-07-11 14:36:17,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase9.apache.org,41517,1689086163885, splitWal=true, meta=true in 4.9820 sec 2023-07-11 14:36:17,594 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1/37e3e6e8b34041bd8bc6a3e3df3c42bd 2023-07-11 14:36:17,595 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(310): Store=f27301259d11533bab9412d981c5782d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:17,595 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region f27301259d11533bab9412d981c5782d 2023-07-11 14:36:17,596 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/norep 2023-07-11 14:36:17,596 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/norep 2023-07-11 14:36:17,597 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f27301259d11533bab9412d981c5782d columnFamilyName norep 2023-07-11 14:36:17,601 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(310): Store=f27301259d11533bab9412d981c5782d/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:17,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d 2023-07-11 14:36:17,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d 2023-07-11 14:36:17,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-11 14:36:17,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:17,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened f27301259d11533bab9412d981c5782d; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68740964, jitterRate=0.02432018518447876}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-11 14:36:17,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for f27301259d11533bab9412d981c5782d: 2023-07-11 14:36:17,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d., pid=25, masterSystemTime=1689086177534 2023-07-11 14:36:17,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:17,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:17,611 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=f27301259d11533bab9412d981c5782d, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:17,611 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086177610"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086177610"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086177610"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086177610"}]},"ts":"1689086177610"} 2023-07-11 14:36:17,615 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-11 14:36:17,615 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure f27301259d11533bab9412d981c5782d, server=jenkins-hbase9.apache.org,42891,1689086172361 in 231 msec 2023-07-11 14:36:17,618 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=17 2023-07-11 14:36:17,618 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase9.apache.org,42167,1689086163940 after splitting done 2023-07-11 14:36:17,618 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN in 391 msec 2023-07-11 14:36:17,618 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase9.apache.org,42167,1689086163940 from processing; numProcessing=0 2023-07-11 14:36:17,620 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase9.apache.org,42167,1689086163940, splitWal=true, meta=false in 5.0120 sec 2023-07-11 14:36:17,792 INFO [jenkins-hbase9:42891Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:18,209 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-11 14:36:18,224 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-11 14:36:18,226 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-11 14:36:18,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.819sec 2023-07-11 14:36:18,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-11 14:36:18,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-11 14:36:18,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-11 14:36:18,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,32785,1689086172324-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-11 14:36:18,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,32785,1689086172324-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-11 14:36:18,230 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-11 14:36:18,299 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(139): Connect 0x2e0b7c70 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:18,304 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3354bdc0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:18,308 DEBUG [hconnection-0x5143ddba-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:18,310 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:33418, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:18,314 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-11 14:36:18,314 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2e0b7c70 to 127.0.0.1:64930 2023-07-11 14:36:18,314 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:18,315 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase9.apache.org:32785 after: jenkins-hbase9.apache.org:32785 2023-07-11 14:36:18,316 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(139): Connect 0x336f042d to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:18,324 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7c0ea1f7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:18,740 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-11 14:36:18,740 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-11 14:36:18,741 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-11 14:36:18,741 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-11 14:36:18,826 DEBUG [hconnection-0x5cbb5af8-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:18,833 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:33432, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:18,859 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689086178859 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:18,865 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication16890861788590x0, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:18,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689086178859-0x10154c21b790022 connected Start Replication Server start 2023-07-11 14:36:18,869 INFO [Listener at localhost/42861] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:18,869 INFO [Listener at localhost/42861] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:18,885 INFO [Listener at localhost/42861] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-11 14:36:18,887 DEBUG [Listener at localhost/42861] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:18,897 INFO [Listener at localhost/42861] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-11 14:36:18,902 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase9.apache.org,42359,1689086159924, jenkins-hbase9.apache.org,37647,1689086159883] other RSs: [] 2023-07-11 14:36:18,925 DEBUG [Listener at localhost/42861.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x0308f05b to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:18,931 DEBUG [Listener at localhost/42861.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@6918cb38, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:18,931 INFO [Listener at localhost/42861.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:18,937 DEBUG [Listener at localhost/42861.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:18,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10154c21b790024 connected 2023-07-11 14:36:18,938 INFO [Listener at localhost/42861.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 to cluster=6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:22,792 INFO [jenkins-hbase9:42891Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:22,876 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase9.apache.org,42359,1689086159924/1's WALs to 1689086178869.syncuptool.replication.org,1234,1 2023-07-11 14:36:22,878 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154 with data PBUF\x08\xFD\xAD\x01 2023-07-11 14:36:22,878 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237 with data PBUF\x08\xDFU 2023-07-11 14:36:22,880 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase9.apache.org,42359,1689086159924/1's WALs to 1689086178869.syncuptool.replication.org,1234,1 2023-07-11 14:36:23,045 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-11 14:36:24,882 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,42359,1689086159924] 2023-07-11 14:36:24,882 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,42359,1689086159924] 2023-07-11 14:36:24,897 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,42359,1689086159924] 2023-07-11 14:36:24,897 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase9.apache.org,42359,1689086159924, ReplicationSource: 1, currentBandwidth=0 2023-07-11 14:36:24,897 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:24,917 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924] zookeeper.ReadOnlyZKClient(139): Connect 0x09d430cc to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:24,923 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@2719f7a9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:24,924 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:24,928 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:24,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10154c21b790026 connected 2023-07-11 14:36:24,929 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase9.apache.org,42359,1689086159924 (queues=2) is replicating from cluster=fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 to cluster=6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:24,930 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1 2023-07-11 14:36:24,934 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase9.apache.org,42359,1689086159924, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-11 14:36:24,934 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0 2023-07-11 14:36:24,934 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1 2023-07-11 14:36:24,935 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase9.apache.org,42359,1689086159924, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-11 14:36:24,936 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0 2023-07-11 14:36:24,942 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237 2023-07-11 14:36:24,942 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154 2023-07-11 14:36:24,942 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-11 14:36:24,942 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-11 14:36:24,942 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-11 14:36:24,942 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-11 14:36:24,943 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase9.apache.org,42359,1689086159924 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/3bfd16063a0672430ddd816d03e73f73/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237, lastWalPosition=15173, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-07-11 14:36:24,944 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase9.apache.org,42359,1689086159924 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/ef703c6f28122a7c70d67fbbd278729c/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs/jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154, lastWalPosition=30605, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-07-11 14:36:24,946 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:24,946 INFO [SinkThread-1] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-07-11 14:36:24,957 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:60550, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-11 14:36:24,958 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42891] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-11 14:36:24,959 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=42891] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-11 14:36:24,959 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42891] zookeeper.ReadOnlyZKClient(139): Connect 0x3f442c37 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:24,965 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3183c5af, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:24,967 DEBUG [hconnection-0x35bc4747-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:24,970 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:60552, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-07-11 14:36:24,979 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42891] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-11 14:36:24,979 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,42359,1689086159924] 2023-07-11 14:36:24,981 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0.1689086161237] 2023-07-11 14:36:24,983 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=42891] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-11 14:36:24,983 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,42359,1689086159924] 2023-07-11 14:36:24,984 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase9.apache.org,42359,1689086159924 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-11 14:36:24,984 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0 of peer 1-jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:24,985 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1.1689086162154] 2023-07-11 14:36:24,988 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase9.apache.org,42359,1689086159924 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-11 14:36:24,988 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1 of peer 1-jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:25,084 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:25,087 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086178859-0x10154c21b790022, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689086178869.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:25,087 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-0,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase9.apache.org,42359,1689086159924 with the following stats: Total replicated edits: 2, current progress: 2023-07-11 14:36:25,187 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase9.apache.org,42359,1689086159924 2023-07-11 14:36:25,188 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] zookeeper.ZKUtil(243): syncupReplication1689086178859-0x10154c21b790022, quorum=127.0.0.1:64930, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689086178869.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,42359,1689086159924 because node does not exist (not an error) 2023-07-11 14:36:25,188 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924.replicationSource.shipperjenkins-hbase9.apache.org%2C42359%2C1689086159924.jenkins-hbase9.apache.org%2C42359%2C1689086159924.regiongroup-1,1-jenkins-hbase9.apache.org,42359,1689086159924] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase9.apache.org,42359,1689086159924 with the following stats: Total replicated edits: 2, current progress: 2023-07-11 14:36:27,792 INFO [jenkins-hbase9:42891Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:27,792 INFO [jenkins-hbase9:42891Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-11 14:36:28,431 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase9.apache.org,37647,1689086159883/1's WALs to 1689086178869.syncuptool.replication.org,1234,1 2023-07-11 14:36:28,433 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0.1689086161243 with data 2023-07-11 14:36:28,435 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase9.apache.org,37647,1689086159883/1's WALs to 1689086178869.syncuptool.replication.org,1234,1 2023-07-11 14:36:30,436 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,37647,1689086159883] 2023-07-11 14:36:30,437 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,37647,1689086159883] 2023-07-11 14:36:30,447 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,37647,1689086159883] 2023-07-11 14:36:30,447 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase9.apache.org,37647,1689086159883, ReplicationSource: 1, currentBandwidth=0 2023-07-11 14:36:30,447 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:30,468 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883] zookeeper.ReadOnlyZKClient(139): Connect 0x1dfecc23 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:30,472 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@a4832e3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:30,472 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:30,475 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:30,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10154c21b790029 connected 2023-07-11 14:36:30,476 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase9.apache.org,37647,1689086159883 (queues=1) is replicating from cluster=fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 to cluster=6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:30,476 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0 2023-07-11 14:36:30,477 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase9.apache.org,37647,1689086159883, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-11 14:36:30,478 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883.replicationSource.shipperjenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1-jenkins-hbase9.apache.org,37647,1689086159883] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0 2023-07-11 14:36:30,483 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1-jenkins-hbase9.apache.org,37647,1689086159883] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs/jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0.1689086161243 2023-07-11 14:36:30,484 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1-jenkins-hbase9.apache.org,37647,1689086159883] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-11 14:36:30,484 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883.replicationSource.shipperjenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1-jenkins-hbase9.apache.org,37647,1689086159883] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase9.apache.org,37647,1689086159883 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-11 14:36:30,484 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883.replicationSource.shipperjenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1-jenkins-hbase9.apache.org,37647,1689086159883] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0 of peer 1-jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:30,584 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883.replicationSource.shipperjenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1-jenkins-hbase9.apache.org,37647,1689086159883] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:30,585 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883.replicationSource.shipperjenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1-jenkins-hbase9.apache.org,37647,1689086159883] zookeeper.ZKUtil(162): syncupReplication1689086178859-0x10154c21b790022, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1689086178869.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,37647,1689086159883/jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0.1689086161243 2023-07-11 14:36:30,587 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086178859-0x10154c21b790022, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689086178869.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,37647,1689086159883/jenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0.1689086161243 2023-07-11 14:36:30,587 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883.replicationSource.shipperjenkins-hbase9.apache.org%2C37647%2C1689086159883.jenkins-hbase9.apache.org%2C37647%2C1689086159883.regiongroup-0,1-jenkins-hbase9.apache.org,37647,1689086159883] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase9.apache.org,37647,1689086159883 with the following stats: Total replicated edits: 0, current progress: 2023-07-11 14:36:30,587 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086178859-0x10154c21b790022, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1689086178869.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:30,588 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086178859-0x10154c21b790022, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689086178869.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,37647,1689086159883 2023-07-11 14:36:30,589 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-11 14:36:32,792 INFO [jenkins-hbase9:42891Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:32,792 INFO [jenkins-hbase9:42891Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-11 14:36:36,087 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-11 14:36:37,792 INFO [jenkins-hbase9:42891Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:37,792 INFO [jenkins-hbase9:42891Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-11 14:36:38,904 INFO [Listener at localhost/42861] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-11 14:36:39,007 DEBUG [Listener at localhost/42861.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b790024, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:39,008 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0308f05b to 127.0.0.1:64930 2023-07-11 14:36:39,009 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:39,210 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086178859-0x10154c21b790022, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:39,210 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086178859-0x10154c21b790022, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:39,224 INFO [Listener at localhost/42861] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-07-11 14:36:39,225 DEBUG [Listener at localhost/42861] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-07-11 14:36:39,226 INFO [Listener at localhost/42861] client.ConnectionUtils(127): master/jenkins-hbase9:0 server-side Connection retries=45 2023-07-11 14:36:39,226 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:39,226 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:39,226 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:39,226 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:39,227 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:39,227 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:39,228 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:44621 2023-07-11 14:36:39,229 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:39,230 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:39,231 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44621 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:39,234 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:446210x0, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:39,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:44621-0x10154c21b79002a connected 2023-07-11 14:36:39,238 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-11 14:36:39,238 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-11 14:36:39,239 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-11 14:36:39,242 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44621 2023-07-11 14:36:39,242 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44621 2023-07-11 14:36:39,242 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44621 2023-07-11 14:36:39,246 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44621 2023-07-11 14:36:39,246 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44621 2023-07-11 14:36:39,246 INFO [Listener at localhost/42861] master.HMaster(444): hbase.rootdir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d, hbase.cluster.distributed=false 2023-07-11 14:36:39,246 DEBUG [pool-556-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-07-11 14:36:39,257 INFO [Listener at localhost/42861] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=45 2023-07-11 14:36:39,258 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:39,258 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:39,258 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:39,258 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:39,258 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:39,258 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:39,260 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:44721 2023-07-11 14:36:39,260 INFO [Listener at localhost/42861] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-11 14:36:39,263 DEBUG [Listener at localhost/42861] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-11 14:36:39,264 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:39,265 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:39,266 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44721 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:39,268 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:447210x0, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:39,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:44721-0x10154c21b79002b connected 2023-07-11 14:36:39,270 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-11 14:36:39,270 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-11 14:36:39,271 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-11 14:36:39,274 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44721 2023-07-11 14:36:39,274 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44721 2023-07-11 14:36:39,277 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44721 2023-07-11 14:36:39,278 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44721 2023-07-11 14:36:39,279 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44721 2023-07-11 14:36:39,280 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:39,282 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-11 14:36:39,282 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:39,289 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-11 14:36:39,289 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-11 14:36:39,289 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:39,290 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-11 14:36:39,291 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase9.apache.org,44621,1689086199226 from backup master directory 2023-07-11 14:36:39,292 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:39,292 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-11 14:36:39,292 WARN [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:39,292 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:39,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-11 14:36:39,308 INFO [master/jenkins-hbase9:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:39,332 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x004b76e9 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:39,335 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c6cb656, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:39,335 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:39,335 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-11 14:36:39,336 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:39,337 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:39,342 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,46131,1689086158709 to hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,46131,1689086158709-dead as it is dead 2023-07-11 14:36:39,343 INFO [master/jenkins-hbase9:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,46131,1689086158709-dead/jenkins-hbase9.apache.org%2C46131%2C1689086158709.jenkins-hbase9.apache.org%2C46131%2C1689086158709.regiongroup-0.1689086160369 2023-07-11 14:36:39,343 INFO [master/jenkins-hbase9:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,46131,1689086158709-dead/jenkins-hbase9.apache.org%2C46131%2C1689086158709.jenkins-hbase9.apache.org%2C46131%2C1689086158709.regiongroup-0.1689086160369 after 0ms 2023-07-11 14:36:39,344 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,46131,1689086158709-dead/jenkins-hbase9.apache.org%2C46131%2C1689086158709.jenkins-hbase9.apache.org%2C46131%2C1689086158709.regiongroup-0.1689086160369 to hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C46131%2C1689086158709.jenkins-hbase9.apache.org%2C46131%2C1689086158709.regiongroup-0.1689086160369 2023-07-11 14:36:39,344 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,46131,1689086158709-dead 2023-07-11 14:36:39,344 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:39,345 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:39,347 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C44621%2C1689086199226.jenkins-hbase9.apache.org%2C44621%2C1689086199226.regiongroup-0, suffix=, logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,44621,1689086199226, archiveDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/oldWALs, maxLogs=10 2023-07-11 14:36:39,354 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,44621,1689086199226/jenkins-hbase9.apache.org%2C44621%2C1689086199226.jenkins-hbase9.apache.org%2C44621%2C1689086199226.regiongroup-0.1689086199347 2023-07-11 14:36:39,354 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37265,DS-aff66802-3d09-4eb6-949b-2fd2dc490c59,DISK], DatanodeInfoWithStorage[127.0.0.1:37609,DS-7fe6473b-be8e-4ffd-9d69-553584c37202,DISK]] 2023-07-11 14:36:39,354 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:39,354 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:39,355 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:39,355 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:39,358 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:39,358 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-11 14:36:39,359 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-11 14:36:39,365 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d2f4bca8d89c4d48af6329fae0cffa15 2023-07-11 14:36:39,365 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:39,365 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-11 14:36:39,365 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C46131%2C1689086158709.jenkins-hbase9.apache.org%2C46131%2C1689086158709.regiongroup-0.1689086160369 2023-07-11 14:36:39,370 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C46131%2C1689086158709.jenkins-hbase9.apache.org%2C46131%2C1689086158709.regiongroup-0.1689086160369 2023-07-11 14:36:39,371 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C46131%2C1689086158709.jenkins-hbase9.apache.org%2C46131%2C1689086158709.regiongroup-0.1689086160369 2023-07-11 14:36:39,375 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:39,377 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-07-11 14:36:39,379 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62367342, jitterRate=-0.07065418362617493}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-11 14:36:39,379 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:39,379 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-11 14:36:39,381 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-11 14:36:39,381 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-11 14:36:39,381 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-11 14:36:39,381 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-11 14:36:39,385 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-11 14:36:39,386 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-11 14:36:39,386 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-11 14:36:39,386 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-11 14:36:39,386 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-11 14:36:39,386 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-11 14:36:39,387 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-11 14:36:39,387 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-11 14:36:39,387 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-11 14:36:39,388 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-11 14:36:39,389 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,42359,1689086159924, table=hbase:meta, region=1588230740 2023-07-11 14:36:39,391 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-07-11 14:36:39,393 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase9.apache.org,37647,1689086159883 already deleted, retry=false 2023-07-11 14:36:39,393 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase9.apache.org,37647,1689086159883 on jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:39,395 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase9.apache.org,37647,1689086159883, splitWal=true, meta=false 2023-07-11 14:36:39,396 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase9.apache.org,37647,1689086159883 (carryingMeta=false) jenkins-hbase9.apache.org,37647,1689086159883/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@601afd2c[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-11 14:36:39,397 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase9.apache.org,42359,1689086159924 already deleted, retry=false 2023-07-11 14:36:39,397 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase9.apache.org,42359,1689086159924 on jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:39,398 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase9.apache.org,42359,1689086159924, splitWal=true, meta=true 2023-07-11 14:36:39,398 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase9.apache.org,42359,1689086159924 (carryingMeta=true) jenkins-hbase9.apache.org,42359,1689086159924/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@609ad5ae[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-11 14:36:39,408 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-11 14:36:39,408 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-11 14:36:39,409 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-11 14:36:39,409 INFO [master/jenkins-hbase9:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-11 14:36:39,409 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-11 14:36:39,410 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-11 14:36:39,410 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-11 14:36:39,413 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-11 14:36:39,414 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-11 14:36:39,414 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-11 14:36:39,414 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:39,414 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase9.apache.org,44621,1689086199226, sessionid=0x10154c21b79002a, setting cluster-up flag (Was=false) 2023-07-11 14:36:39,420 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-11 14:36:39,421 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:39,423 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-11 14:36:39,424 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:39,425 WARN [master/jenkins-hbase9:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/.hbase-snapshot/.tmp 2023-07-11 14:36:39,426 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:39,426 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:39,426 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:39,426 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:39,426 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase9:0, corePoolSize=10, maxPoolSize=10 2023-07-11 14:36:39,426 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,427 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:39,427 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,433 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689086229433 2023-07-11 14:36:39,434 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-11 14:36:39,434 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-11 14:36:39,434 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-11 14:36:39,434 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-11 14:36:39,434 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-11 14:36:39,434 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-11 14:36:39,434 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:39,435 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase9.apache.org,42359,1689086159924; numProcessing=1 2023-07-11 14:36:39,435 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,42359,1689086159924, splitWal=true, meta=true 2023-07-11 14:36:39,435 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-11 14:36:39,436 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-11 14:36:39,436 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-11 14:36:39,436 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-11 14:36:39,437 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-11 14:36:39,437 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase9.apache.org,37647,1689086159883; numProcessing=2 2023-07-11 14:36:39,437 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,37647,1689086159883, splitWal=true, meta=false 2023-07-11 14:36:39,437 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086199437,5,FailOnTimeoutGroup] 2023-07-11 14:36:39,437 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,42359,1689086159924, splitWal=true, meta=true, isMeta: true 2023-07-11 14:36:39,442 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086199437,5,FailOnTimeoutGroup] 2023-07-11 14:36:39,442 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:39,442 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-11 14:36:39,442 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:39,442 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:39,442 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689086199442, completionTime=-1 2023-07-11 14:36:39,442 WARN [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-11 14:36:39,442 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-11 14:36:39,443 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924-splitting 2023-07-11 14:36:39,444 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924-splitting dir is empty, no logs to split. 2023-07-11 14:36:39,444 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase9.apache.org,42359,1689086159924 WAL count=0, meta=true 2023-07-11 14:36:39,447 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924-splitting dir is empty, no logs to split. 2023-07-11 14:36:39,447 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase9.apache.org,42359,1689086159924 WAL count=0, meta=true 2023-07-11 14:36:39,447 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase9.apache.org,42359,1689086159924 WAL splitting is done? wals=0, meta=true 2023-07-11 14:36:39,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-11 14:36:39,449 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-11 14:36:39,451 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:39,480 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(951): ClusterId : fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 2023-07-11 14:36:39,480 DEBUG [RS:0;jenkins-hbase9:44721] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-11 14:36:39,482 DEBUG [RS:0;jenkins-hbase9:44721] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-11 14:36:39,482 DEBUG [RS:0;jenkins-hbase9:44721] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-11 14:36:39,485 DEBUG [RS:0;jenkins-hbase9:44721] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-11 14:36:39,487 DEBUG [RS:0;jenkins-hbase9:44721] zookeeper.ReadOnlyZKClient(139): Connect 0x7864cd48 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:39,490 DEBUG [RS:0;jenkins-hbase9:44721] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@76a33c34, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:39,490 DEBUG [RS:0;jenkins-hbase9:44721] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@729ca638, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:39,499 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase9:44721 2023-07-11 14:36:39,499 INFO [RS:0;jenkins-hbase9:44721] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-11 14:36:39,499 INFO [RS:0;jenkins-hbase9:44721] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-11 14:36:39,499 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1022): About to register with Master. 2023-07-11 14:36:39,499 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase9.apache.org,44621,1689086199226 with isa=jenkins-hbase9.apache.org/172.31.2.10:44721, startcode=1689086199257 2023-07-11 14:36:39,499 DEBUG [RS:0;jenkins-hbase9:44721] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-11 14:36:39,504 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:50647, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-11 14:36:39,505 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44621] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:39,505 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d 2023-07-11 14:36:39,505 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41201 2023-07-11 14:36:39,505 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-11 14:36:39,510 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-11 14:36:39,511 DEBUG [RS:0;jenkins-hbase9:44721] zookeeper.ZKUtil(162): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:39,511 WARN [RS:0;jenkins-hbase9:44721] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:39,511 INFO [RS:0;jenkins-hbase9:44721] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:39,511 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,44721,1689086199257] 2023-07-11 14:36:39,512 INFO [RS:0;jenkins-hbase9:44721] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:39,512 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:39,526 INFO [RS:0;jenkins-hbase9:44721] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-11 14:36:39,526 DEBUG [RS:0;jenkins-hbase9:44721] zookeeper.ZKUtil(162): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:39,527 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:39,527 INFO [RS:0;jenkins-hbase9:44721] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-11 14:36:39,528 INFO [RS:0;jenkins-hbase9:44721] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-11 14:36:39,529 INFO [RS:0;jenkins-hbase9:44721] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-11 14:36:39,529 INFO [RS:0;jenkins-hbase9:44721] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:39,529 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-11 14:36:39,530 INFO [RS:0;jenkins-hbase9:44721] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:39,531 DEBUG [RS:0;jenkins-hbase9:44721] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,531 DEBUG [RS:0;jenkins-hbase9:44721] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,531 DEBUG [RS:0;jenkins-hbase9:44721] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,531 DEBUG [RS:0;jenkins-hbase9:44721] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,531 DEBUG [RS:0;jenkins-hbase9:44721] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,531 DEBUG [RS:0;jenkins-hbase9:44721] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:39,531 DEBUG [RS:0;jenkins-hbase9:44721] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,531 DEBUG [RS:0;jenkins-hbase9:44721] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,531 DEBUG [RS:0;jenkins-hbase9:44721] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,531 DEBUG [RS:0;jenkins-hbase9:44721] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:39,532 INFO [RS:0;jenkins-hbase9:44721] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:39,532 INFO [RS:0;jenkins-hbase9:44721] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:39,532 INFO [RS:0;jenkins-hbase9:44721] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:39,542 INFO [RS:0;jenkins-hbase9:44721] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-11 14:36:39,542 INFO [RS:0;jenkins-hbase9:44721] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44721,1689086199257-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:39,543 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=101ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-11 14:36:39,552 INFO [RS:0;jenkins-hbase9:44721] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-11 14:36:39,553 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1689086178869.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase9.apache.org,44721,1689086199257] 2023-07-11 14:36:39,562 INFO [RS:0;jenkins-hbase9:44721] regionserver.Replication(203): jenkins-hbase9.apache.org,44721,1689086199257 started 2023-07-11 14:36:39,562 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1637): Serving as jenkins-hbase9.apache.org,44721,1689086199257, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:44721, sessionid=0x10154c21b79002b 2023-07-11 14:36:39,562 DEBUG [RS:0;jenkins-hbase9:44721] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-11 14:36:39,562 DEBUG [RS:0;jenkins-hbase9:44721] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:39,562 DEBUG [RS:0;jenkins-hbase9:44721] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,44721,1689086199257' 2023-07-11 14:36:39,562 DEBUG [RS:0;jenkins-hbase9:44721] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-11 14:36:39,563 DEBUG [RS:0;jenkins-hbase9:44721] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-11 14:36:39,563 DEBUG [RS:0;jenkins-hbase9:44721] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-11 14:36:39,563 DEBUG [RS:0;jenkins-hbase9:44721] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-11 14:36:39,563 DEBUG [RS:0;jenkins-hbase9:44721] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:39,563 DEBUG [RS:0;jenkins-hbase9:44721] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,44721,1689086199257' 2023-07-11 14:36:39,563 DEBUG [RS:0;jenkins-hbase9:44721] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-11 14:36:39,564 DEBUG [RS:0;jenkins-hbase9:44721] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-11 14:36:39,564 DEBUG [RS:0;jenkins-hbase9:44721] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-11 14:36:39,564 INFO [RS:0;jenkins-hbase9:44721] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-11 14:36:39,564 INFO [RS:0;jenkins-hbase9:44721] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-11 14:36:39,573 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x5c214aec to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:39,577 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4fe6d45a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:39,578 INFO [RS:0;jenkins-hbase9:44721.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:39,579 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:39,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10154c21b79002f connected 2023-07-11 14:36:39,583 INFO [RS:0;jenkins-hbase9:44721.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 to cluster=6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:39,601 DEBUG [jenkins-hbase9:44621] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-11 14:36:39,602 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,44721,1689086199257, state=OPENING 2023-07-11 14:36:39,603 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-11 14:36:39,603 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-11 14:36:39,603 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,44721,1689086199257}] 2023-07-11 14:36:39,664 INFO [RS:0;jenkins-hbase9:44721] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:39,666 INFO [RS:0;jenkins-hbase9:44721] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0, suffix=, logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257, archiveDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs, maxLogs=10 2023-07-11 14:36:39,682 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0 for peer 1 2023-07-11 14:36:39,682 INFO [RS:0;jenkins-hbase9:44721] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0.1689086199666 2023-07-11 14:36:39,682 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0 2023-07-11 14:36:39,682 INFO [RS:0;jenkins-hbase9:44721] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-11 14:36:39,683 DEBUG [RS:0;jenkins-hbase9:44721] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37609,DS-7fe6473b-be8e-4ffd-9d69-553584c37202,DISK], DatanodeInfoWithStorage[127.0.0.1:37265,DS-aff66802-3d09-4eb6-949b-2fd2dc490c59,DISK]] 2023-07-11 14:36:39,683 INFO [RS:0;jenkins-hbase9:44721.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0 2023-07-11 14:36:39,688 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:39,759 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:39,759 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:39,761 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:46700, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-11 14:36:39,764 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-11 14:36:39,765 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:39,765 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:39,765 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:39,766 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C44721%2C1689086199257.meta, suffix=.meta, logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257, archiveDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs, maxLogs=10 2023-07-11 14:36:39,780 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.meta.1689086199767.meta 2023-07-11 14:36:39,780 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37265,DS-aff66802-3d09-4eb6-949b-2fd2dc490c59,DISK], DatanodeInfoWithStorage[127.0.0.1:37609,DS-7fe6473b-be8e-4ffd-9d69-553584c37202,DISK]] 2023-07-11 14:36:39,780 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:39,781 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-11 14:36:39,781 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-11 14:36:39,781 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-11 14:36:39,781 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-11 14:36:39,781 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:39,781 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-11 14:36:39,781 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-11 14:36:39,783 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-11 14:36:39,784 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/info 2023-07-11 14:36:39,784 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/info 2023-07-11 14:36:39,784 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-11 14:36:39,792 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/info/aca6c0b18e134ac29b816a82dc14ccf7 2023-07-11 14:36:39,794 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:39,794 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-11 14:36:39,795 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:39,795 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:39,795 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-11 14:36:39,795 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:39,805 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/rep_barrier/1d3276f9ba23404ab9348dc7a8b8091a 2023-07-11 14:36:39,806 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:39,806 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-11 14:36:39,807 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/table 2023-07-11 14:36:39,807 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/table 2023-07-11 14:36:39,807 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-11 14:36:39,814 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/table/e62ac9c81e8e434e8393ec6d5abd21c0 2023-07-11 14:36:39,814 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:39,815 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740 2023-07-11 14:36:39,817 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740 2023-07-11 14:36:39,819 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-11 14:36:39,820 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-11 14:36:39,821 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71800617, jitterRate=0.06991256773471832}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-11 14:36:39,821 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-11 14:36:39,824 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1689086199759 2023-07-11 14:36:39,830 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-11 14:36:39,831 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-11 14:36:39,831 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,44721,1689086199257, state=OPEN 2023-07-11 14:36:39,832 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-11 14:36:39,833 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-11 14:36:39,835 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-07-11 14:36:39,835 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,44721,1689086199257 in 230 msec 2023-07-11 14:36:39,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-11 14:36:39,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 387 msec 2023-07-11 14:36:39,901 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:40,006 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:40,112 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:40,218 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:40,324 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:40,430 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:40,535 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:40,640 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:40,745 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:40,851 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:40,956 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:41,046 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1604ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-11 14:36:41,061 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:41,166 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:41,271 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:41,377 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:41,482 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:41,587 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:41,692 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:41,797 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:41,903 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:42,008 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:42,113 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:42,219 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:42,323 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:42,428 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:42,533 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:42,548 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3106ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-11 14:36:42,638 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:42,743 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:42,792 INFO [jenkins-hbase9:42891Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:42,792 INFO [jenkins-hbase9:42891Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-11 14:36:42,848 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:42,953 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:43,058 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:43,162 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:43,267 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:43,372 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:43,477 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:43,582 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:43,687 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:43,791 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:43,896 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:43,951 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4509ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-11 14:36:43,951 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-11 14:36:43,953 DEBUG [hconnection-0x4a5644a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:43,955 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45384, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:43,960 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=11796b6d2ba24476446363b9a5701d72, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,42359,1689086159924, regionLocation=jenkins-hbase9.apache.org,42359,1689086159924, openSeqNum=2 2023-07-11 14:36:43,961 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=3bfd16063a0672430ddd816d03e73f73, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,42359,1689086159924, regionLocation=jenkins-hbase9.apache.org,42359,1689086159924, openSeqNum=2 2023-07-11 14:36:43,961 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ef703c6f28122a7c70d67fbbd278729c, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,42359,1689086159924, regionLocation=jenkins-hbase9.apache.org,42359,1689086159924, openSeqNum=2 2023-07-11 14:36:43,961 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-11 14:36:43,961 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689086263961 2023-07-11 14:36:43,961 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689086323961 2023-07-11 14:36:43,961 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 10 msec 2023-07-11 14:36:43,977 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase9.apache.org,42359,1689086159924 had 4 regions 2023-07-11 14:36:43,977 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase9.apache.org,37647,1689086159883 had 0 regions 2023-07-11 14:36:43,977 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44621,1689086199226-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:43,978 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44621,1689086199226-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:43,978 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44621,1689086199226-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:43,978 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase9:44621, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:43,978 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:43,978 WARN [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. is NOT online; state={11796b6d2ba24476446363b9a5701d72 state=OPEN, ts=1689086203960, server=jenkins-hbase9.apache.org,42359,1689086159924}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-11 14:36:43,979 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,37647,1689086159883, splitWal=true, meta=false, isMeta: false 2023-07-11 14:36:43,980 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,42359,1689086159924, splitWal=true, meta=true, isMeta: false 2023-07-11 14:36:43,981 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,37647,1689086159883-splitting 2023-07-11 14:36:43,982 WARN [master/jenkins-hbase9:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase9.apache.org,42359,1689086159924/hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72., unknown_server=jenkins-hbase9.apache.org,42359,1689086159924/t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73., unknown_server=jenkins-hbase9.apache.org,42359,1689086159924/t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:43,982 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,37647,1689086159883-splitting dir is empty, no logs to split. 2023-07-11 14:36:43,982 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase9.apache.org,37647,1689086159883 WAL count=0, meta=false 2023-07-11 14:36:43,983 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924-splitting dir is empty, no logs to split. 2023-07-11 14:36:43,983 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase9.apache.org,42359,1689086159924 WAL count=0, meta=false 2023-07-11 14:36:43,985 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,37647,1689086159883-splitting dir is empty, no logs to split. 2023-07-11 14:36:43,985 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase9.apache.org,37647,1689086159883 WAL count=0, meta=false 2023-07-11 14:36:43,985 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase9.apache.org,37647,1689086159883 WAL splitting is done? wals=0, meta=false 2023-07-11 14:36:43,986 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,42359,1689086159924-splitting dir is empty, no logs to split. 2023-07-11 14:36:43,986 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase9.apache.org,42359,1689086159924 WAL count=0, meta=false 2023-07-11 14:36:43,986 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase9.apache.org,42359,1689086159924 WAL splitting is done? wals=0, meta=false 2023-07-11 14:36:43,987 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase9.apache.org,37647,1689086159883 failed, ignore...File hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,37647,1689086159883-splitting does not exist. 2023-07-11 14:36:43,987 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=3bfd16063a0672430ddd816d03e73f73, ASSIGN}, {pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ef703c6f28122a7c70d67fbbd278729c, ASSIGN}, {pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=11796b6d2ba24476446363b9a5701d72, ASSIGN}] 2023-07-11 14:36:43,988 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase9.apache.org,37647,1689086159883 after splitting done 2023-07-11 14:36:43,988 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase9.apache.org,37647,1689086159883 from processing; numProcessing=1 2023-07-11 14:36:43,988 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=3bfd16063a0672430ddd816d03e73f73, ASSIGN 2023-07-11 14:36:43,989 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ef703c6f28122a7c70d67fbbd278729c, ASSIGN 2023-07-11 14:36:43,989 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=11796b6d2ba24476446363b9a5701d72, ASSIGN 2023-07-11 14:36:43,990 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=3bfd16063a0672430ddd816d03e73f73, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:43,990 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=ef703c6f28122a7c70d67fbbd278729c, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:43,990 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=11796b6d2ba24476446363b9a5701d72, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:43,990 DEBUG [jenkins-hbase9:44621] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-11 14:36:43,992 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase9.apache.org,37647,1689086159883, splitWal=true, meta=false in 4.5950 sec 2023-07-11 14:36:43,993 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=3bfd16063a0672430ddd816d03e73f73, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:43,993 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=ef703c6f28122a7c70d67fbbd278729c, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:43,993 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=11796b6d2ba24476446363b9a5701d72, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:43,993 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086203993"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086203993"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086203993"}]},"ts":"1689086203993"} 2023-07-11 14:36:43,993 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086203993"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086203993"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086203993"}]},"ts":"1689086203993"} 2023-07-11 14:36:43,993 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086203993"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086203993"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086203993"}]},"ts":"1689086203993"} 2023-07-11 14:36:43,995 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=23, state=RUNNABLE; OpenRegionProcedure ef703c6f28122a7c70d67fbbd278729c, server=jenkins-hbase9.apache.org,44721,1689086199257}] 2023-07-11 14:36:43,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=22, state=RUNNABLE; OpenRegionProcedure 3bfd16063a0672430ddd816d03e73f73, server=jenkins-hbase9.apache.org,44721,1689086199257}] 2023-07-11 14:36:43,997 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=24, state=RUNNABLE; OpenRegionProcedure 11796b6d2ba24476446363b9a5701d72, server=jenkins-hbase9.apache.org,44721,1689086199257}] 2023-07-11 14:36:44,002 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:44,107 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:44,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:44,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:44,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1, suffix=, logDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257, archiveDir=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs, maxLogs=10 2023-07-11 14:36:44,156 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:44,156 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 11796b6d2ba24476446363b9a5701d72, NAME => 'hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:44,156 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:44,156 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:44,156 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:44,156 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:44,158 INFO [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:44,159 DEBUG [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/info 2023-07-11 14:36:44,159 DEBUG [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/info 2023-07-11 14:36:44,160 INFO [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11796b6d2ba24476446363b9a5701d72 columnFamilyName info 2023-07-11 14:36:44,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1 for peer 1 2023-07-11 14:36:44,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1.1689086204154 2023-07-11 14:36:44,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1 2023-07-11 14:36:44,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-11 14:36:44,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37265,DS-aff66802-3d09-4eb6-949b-2fd2dc490c59,DISK], DatanodeInfoWithStorage[127.0.0.1:37609,DS-7fe6473b-be8e-4ffd-9d69-553584c37202,DISK]] 2023-07-11 14:36:44,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1 2023-07-11 14:36:44,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3bfd16063a0672430ddd816d03e73f73, NAME => 't1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:44,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:44,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:44,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:44,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:44,167 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:44,168 DEBUG [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/cf1 2023-07-11 14:36:44,168 DEBUG [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/cf1 2023-07-11 14:36:44,168 DEBUG [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] regionserver.HStore(539): loaded hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/info/d80a718065494749a24f9a2f7bd919d2 2023-07-11 14:36:44,169 INFO [StoreOpener-11796b6d2ba24476446363b9a5701d72-1] regionserver.HStore(310): Store=11796b6d2ba24476446363b9a5701d72/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:44,169 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bfd16063a0672430ddd816d03e73f73 columnFamilyName cf1 2023-07-11 14:36:44,170 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:44,170 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-11 14:36:44,171 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:44,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:44,175 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 11796b6d2ba24476446363b9a5701d72; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61416333, jitterRate=-0.08482532203197479}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-11 14:36:44,175 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 11796b6d2ba24476446363b9a5701d72: 2023-07-11 14:36:44,175 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08392405cee649fd8d2ff517b975a7cd 2023-07-11 14:36:44,175 DEBUG [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] regionserver.HStore(539): loaded hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/cf1/08392405cee649fd8d2ff517b975a7cd 2023-07-11 14:36:44,179 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] regionserver.HStore(310): Store=3bfd16063a0672430ddd816d03e73f73/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:44,179 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:44,180 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72., pid=27, masterSystemTime=1689086204147 2023-07-11 14:36:44,181 DEBUG [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/norep 2023-07-11 14:36:44,181 DEBUG [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/norep 2023-07-11 14:36:44,182 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bfd16063a0672430ddd816d03e73f73 columnFamilyName norep 2023-07-11 14:36:44,183 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:44,183 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:44,184 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=11796b6d2ba24476446363b9a5701d72, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:44,184 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086204184"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086204184"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086204184"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086204184"}]},"ts":"1689086204184"} 2023-07-11 14:36:44,188 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=24 2023-07-11 14:36:44,188 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=24, state=SUCCESS; OpenRegionProcedure 11796b6d2ba24476446363b9a5701d72, server=jenkins-hbase9.apache.org,44721,1689086199257 in 189 msec 2023-07-11 14:36:44,190 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=11796b6d2ba24476446363b9a5701d72, ASSIGN in 201 msec 2023-07-11 14:36:44,192 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68eebf33a44e4b408009b187181e030d 2023-07-11 14:36:44,192 DEBUG [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] regionserver.HStore(539): loaded hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/norep/68eebf33a44e4b408009b187181e030d 2023-07-11 14:36:44,193 INFO [StoreOpener-3bfd16063a0672430ddd816d03e73f73-1] regionserver.HStore(310): Store=3bfd16063a0672430ddd816d03e73f73/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:44,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:44,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:44,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-11 14:36:44,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:44,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 3bfd16063a0672430ddd816d03e73f73; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67858035, jitterRate=0.011163517832756042}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-11 14:36:44,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 3bfd16063a0672430ddd816d03e73f73: 2023-07-11 14:36:44,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73., pid=26, masterSystemTime=1689086204147 2023-07-11 14:36:44,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:44,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:44,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:44,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ef703c6f28122a7c70d67fbbd278729c, NAME => 't2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:44,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:44,203 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=3bfd16063a0672430ddd816d03e73f73, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:44,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:44,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:44,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:44,204 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086204203"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086204203"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086204203"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086204203"}]},"ts":"1689086204203"} 2023-07-11 14:36:44,205 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:44,206 DEBUG [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/cf1 2023-07-11 14:36:44,206 DEBUG [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/cf1 2023-07-11 14:36:44,207 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ef703c6f28122a7c70d67fbbd278729c columnFamilyName cf1 2023-07-11 14:36:44,208 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=22 2023-07-11 14:36:44,208 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=22, state=SUCCESS; OpenRegionProcedure 3bfd16063a0672430ddd816d03e73f73, server=jenkins-hbase9.apache.org,44721,1689086199257 in 210 msec 2023-07-11 14:36:44,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=3bfd16063a0672430ddd816d03e73f73, ASSIGN in 221 msec 2023-07-11 14:36:44,251 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:44,251 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-11 14:36:44,251 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:44,251 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0.1689086199666, lastWalPosition=422, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-11 14:36:44,253 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a074c76829194557b0b7256c8de40f99 2023-07-11 14:36:44,253 DEBUG [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] regionserver.HStore(539): loaded hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/cf1/a074c76829194557b0b7256c8de40f99 2023-07-11 14:36:44,253 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] regionserver.HStore(310): Store=ef703c6f28122a7c70d67fbbd278729c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:44,253 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:44,254 DEBUG [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/norep 2023-07-11 14:36:44,254 DEBUG [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/norep 2023-07-11 14:36:44,255 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ef703c6f28122a7c70d67fbbd278729c columnFamilyName norep 2023-07-11 14:36:44,261 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 698a73b014474ad6a0a2f76f3101447c 2023-07-11 14:36:44,261 DEBUG [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] regionserver.HStore(539): loaded hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/norep/698a73b014474ad6a0a2f76f3101447c 2023-07-11 14:36:44,261 INFO [StoreOpener-ef703c6f28122a7c70d67fbbd278729c-1] regionserver.HStore(310): Store=ef703c6f28122a7c70d67fbbd278729c/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:44,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:44,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:44,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-11 14:36:44,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:44,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened ef703c6f28122a7c70d67fbbd278729c; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63492354, jitterRate=-0.05389019846916199}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-11 14:36:44,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for ef703c6f28122a7c70d67fbbd278729c: 2023-07-11 14:36:44,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c., pid=25, masterSystemTime=1689086204147 2023-07-11 14:36:44,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:44,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:44,270 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=ef703c6f28122a7c70d67fbbd278729c, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:44,270 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086204270"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086204270"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086204270"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086204270"}]},"ts":"1689086204270"} 2023-07-11 14:36:44,274 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=23 2023-07-11 14:36:44,274 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=23, state=SUCCESS; OpenRegionProcedure ef703c6f28122a7c70d67fbbd278729c, server=jenkins-hbase9.apache.org,44721,1689086199257 in 277 msec 2023-07-11 14:36:44,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:44,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-11 14:36:44,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:44,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1.1689086204154, lastWalPosition=823, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-11 14:36:44,277 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=19 2023-07-11 14:36:44,277 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=ef703c6f28122a7c70d67fbbd278729c, ASSIGN in 287 msec 2023-07-11 14:36:44,277 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase9.apache.org,42359,1689086159924 after splitting done 2023-07-11 14:36:44,277 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase9.apache.org,42359,1689086159924 from processing; numProcessing=0 2023-07-11 14:36:44,278 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase9.apache.org,42359,1689086159924, splitWal=true, meta=true in 4.8800 sec 2023-07-11 14:36:44,356 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:44,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:44,461 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:44,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:44,512 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-11 14:36:44,562 INFO [jenkins-hbase9:44721Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1]: currently replicating from: hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1.1689086204154 at position: 823 walGroup [jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0]: currently replicating from: hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0.1689086199666 at position: 422 2023-07-11 14:36:44,572 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:44,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:44,677 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:44,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:44,783 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:44,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:44,888 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:44,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:44,979 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-07-11 14:36:44,993 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:44,994 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-11 14:36:44,996 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-11 14:36:44,996 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.704sec 2023-07-11 14:36:44,997 INFO [master/jenkins-hbase9:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-11 14:36:44,997 INFO [master/jenkins-hbase9:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-11 14:36:44,997 INFO [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-11 14:36:44,997 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44621,1689086199226-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-11 14:36:44,997 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44621,1689086199226-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-11 14:36:45,000 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-11 14:36:45,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:45,084 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(139): Connect 0x1bc6830a to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:45,089 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3e5a0a21, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:45,091 DEBUG [hconnection-0x595a7176-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:45,092 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45388, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:45,099 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-11 14:36:45,099 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1bc6830a to 127.0.0.1:64930 2023-07-11 14:36:45,099 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:45,101 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase9.apache.org:44621 after: jenkins-hbase9.apache.org:44621 2023-07-11 14:36:45,101 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(139): Connect 0x11ec8647 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:45,101 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:45,109 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1bb794cd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:45,110 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x336f042d to 127.0.0.1:64930 2023-07-11 14:36:45,110 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:45,110 DEBUG [Listener at localhost/42861] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-11 14:36:45,110 DEBUG [Listener at localhost/42861] util.JVMClusterUtil(257): Found active master hash=119381726, stopped=false 2023-07-11 14:36:45,110 INFO [Listener at localhost/42861] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:45,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:45,118 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-11 14:36:45,118 INFO [Listener at localhost/42861] procedure2.ProcedureExecutor(629): Stopping 2023-07-11 14:36:45,118 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-11 14:36:45,118 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:45,118 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x433cbd50 to 127.0.0.1:64930 2023-07-11 14:36:45,120 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:45,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:45,120 INFO [Listener at localhost/42861] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,42891,1689086172361' ***** 2023-07-11 14:36:45,121 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:45,121 INFO [Listener at localhost/42861] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-11 14:36:45,121 INFO [RS:0;jenkins-hbase9:42891] regionserver.HeapMemoryManager(220): Stopping 2023-07-11 14:36:45,121 INFO [RS:0;jenkins-hbase9:42891] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-11 14:36:45,121 INFO [RS:0;jenkins-hbase9:42891] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-11 14:36:45,121 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(3305): Received CLOSE for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:45,122 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-11 14:36:45,122 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(3305): Received CLOSE for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:45,122 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(3305): Received CLOSE for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:45,122 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:45,122 DEBUG [RS:0;jenkins-hbase9:42891] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x27c81d4b to 127.0.0.1:64930 2023-07-11 14:36:45,122 DEBUG [RS:0;jenkins-hbase9:42891] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:45,122 INFO [RS:0;jenkins-hbase9:42891] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-11 14:36:45,122 INFO [RS:0;jenkins-hbase9:42891] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-11 14:36:45,122 INFO [RS:0;jenkins-hbase9:42891] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-11 14:36:45,122 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-11 14:36:45,122 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-11 14:36:45,122 DEBUG [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1478): Online Regions={f27301259d11533bab9412d981c5782d=t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d., 1588230740=hbase:meta,,1.1588230740, d4185bdf44467f073faaa5de49a197d7=hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7., 34f512aa6fa4c9880527ac501e69b8b2=t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.} 2023-07-11 14:36:45,122 DEBUG [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1504): Waiting on 1588230740, 34f512aa6fa4c9880527ac501e69b8b2, d4185bdf44467f073faaa5de49a197d7, f27301259d11533bab9412d981c5782d 2023-07-11 14:36:45,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing f27301259d11533bab9412d981c5782d, disabling compactions & flushes 2023-07-11 14:36:45,125 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-11 14:36:45,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:45,125 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-11 14:36:45,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:45,125 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-11 14:36:45,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. after waiting 0 ms 2023-07-11 14:36:45,125 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-11 14:36:45,126 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-11 14:36:45,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:45,126 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-11 14:36:45,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing f27301259d11533bab9412d981c5782d 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-07-11 14:36:45,166 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/.tmp/info/1041948364674772bb741eb2149d53de 2023-07-11 14:36:45,181 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:45,184 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/.tmp/info/1041948364674772bb741eb2149d53de as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info/1041948364674772bb741eb2149d53de 2023-07-11 14:36:45,191 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info/1041948364674772bb741eb2149d53de, entries=24, sequenceid=31, filesize=7.4 K 2023-07-11 14:36:45,195 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 69ms, sequenceid=31, compaction requested=false 2023-07-11 14:36:45,202 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/.tmp/cf1/969a0b4d2c414fd78805717f8ef78708 2023-07-11 14:36:45,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 969a0b4d2c414fd78805717f8ef78708 2023-07-11 14:36:45,225 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:45,226 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/.tmp/cf1/969a0b4d2c414fd78805717f8ef78708 as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1/969a0b4d2c414fd78805717f8ef78708 2023-07-11 14:36:45,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:45,228 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-11 14:36:45,229 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-11 14:36:45,230 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-11 14:36:45,230 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-11 14:36:45,230 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-11 14:36:45,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 969a0b4d2c414fd78805717f8ef78708 2023-07-11 14:36:45,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1/969a0b4d2c414fd78805717f8ef78708, entries=50, sequenceid=14, filesize=6.4 K 2023-07-11 14:36:45,237 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for f27301259d11533bab9412d981c5782d in 111ms, sequenceid=14, compaction requested=false 2023-07-11 14:36:45,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-07-11 14:36:45,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:45,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for f27301259d11533bab9412d981c5782d: 2023-07-11 14:36:45,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:45,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing d4185bdf44467f073faaa5de49a197d7, disabling compactions & flushes 2023-07-11 14:36:45,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:45,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:45,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. after waiting 0 ms 2023-07-11 14:36:45,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:45,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-11 14:36:45,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:45,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for d4185bdf44467f073faaa5de49a197d7: 2023-07-11 14:36:45,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:45,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 34f512aa6fa4c9880527ac501e69b8b2, disabling compactions & flushes 2023-07-11 14:36:45,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:45,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:45,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. after waiting 0 ms 2023-07-11 14:36:45,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:45,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 34f512aa6fa4c9880527ac501e69b8b2 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-07-11 14:36:45,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=15 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/.tmp/cf1/3976bd393a1a41dc89cbea674d1145b4 2023-07-11 14:36:45,314 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3976bd393a1a41dc89cbea674d1145b4 2023-07-11 14:36:45,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/.tmp/cf1/3976bd393a1a41dc89cbea674d1145b4 as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1/3976bd393a1a41dc89cbea674d1145b4 2023-07-11 14:36:45,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3976bd393a1a41dc89cbea674d1145b4 2023-07-11 14:36:45,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1/3976bd393a1a41dc89cbea674d1145b4, entries=100, sequenceid=15, filesize=7.9 K 2023-07-11 14:36:45,322 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for 34f512aa6fa4c9880527ac501e69b8b2 in 52ms, sequenceid=15, compaction requested=false 2023-07-11 14:36:45,323 DEBUG [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1504): Waiting on 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:45,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:45,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/recovered.edits/18.seqid, newMaxSeqId=18, maxSeqId=11 2023-07-11 14:36:45,340 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:45,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 34f512aa6fa4c9880527ac501e69b8b2: 2023-07-11 14:36:45,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:45,340 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:45,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:45,445 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:45,523 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,42891,1689086172361; all regions closed. 2023-07-11 14:36:45,524 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:45,533 DEBUG [RS:0;jenkins-hbase9:42891] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:36:45,533 INFO [RS:0;jenkins-hbase9:42891] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C42891%2C1689086172361.meta:.meta(num 1689086172997) 2023-07-11 14:36:45,533 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:45,541 DEBUG [RS:0;jenkins-hbase9:42891] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:36:45,541 INFO [RS:0;jenkins-hbase9:42891] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C42891%2C1689086172361.jenkins-hbase9.apache.org%2C42891%2C1689086172361.regiongroup-1:(num 1689086177541) 2023-07-11 14:36:45,541 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:45,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:45,552 DEBUG [RS:0;jenkins-hbase9:42891] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:36:45,552 INFO [RS:0;jenkins-hbase9:42891] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C42891%2C1689086172361.jenkins-hbase9.apache.org%2C42891%2C1689086172361.regiongroup-0:(num 1689086172898) 2023-07-11 14:36:45,552 DEBUG [RS:0;jenkins-hbase9:42891] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:45,552 INFO [RS:0;jenkins-hbase9:42891] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:45,552 INFO [RS:0;jenkins-hbase9:42891] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-11 14:36:45,552 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:45,552 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:45,552 DEBUG [RS:0;jenkins-hbase9:42891] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3f442c37 to 127.0.0.1:64930 2023-07-11 14:36:45,553 DEBUG [RS:0;jenkins-hbase9:42891] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:45,554 INFO [RS:0;jenkins-hbase9:42891] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:42891 2023-07-11 14:36:45,557 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,42891,1689086172361 2023-07-11 14:36:45,557 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b790026, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:45,557 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:45,557 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:45,557 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-11 14:36:45,560 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,42891,1689086172361] 2023-07-11 14:36:45,560 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,42891,1689086172361; numProcessing=1 2023-07-11 14:36:45,561 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,42891,1689086172361 already deleted, retry=false 2023-07-11 14:36:45,561 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,42891,1689086172361 expired; onlineServers=0 2023-07-11 14:36:45,561 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,32785,1689086172324' ***** 2023-07-11 14:36:45,561 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-11 14:36:45,561 DEBUG [M:0;jenkins-hbase9:32785] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3221ae5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:45,562 INFO [M:0;jenkins-hbase9:32785] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:45,562 INFO [M:0;jenkins-hbase9:32785] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,32785,1689086172324; all regions closed. 2023-07-11 14:36:45,562 DEBUG [M:0;jenkins-hbase9:32785] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:45,562 DEBUG [M:0;jenkins-hbase9:32785] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-11 14:36:45,562 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-11 14:36:45,562 DEBUG [M:0;jenkins-hbase9:32785] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-11 14:36:45,562 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086172652] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086172652,5,FailOnTimeoutGroup] 2023-07-11 14:36:45,563 INFO [M:0;jenkins-hbase9:32785] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-11 14:36:45,562 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086172651] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086172651,5,FailOnTimeoutGroup] 2023-07-11 14:36:45,563 INFO [M:0;jenkins-hbase9:32785] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-11 14:36:45,564 INFO [M:0;jenkins-hbase9:32785] hbase.ChoreService(369): Chore service for: master/jenkins-hbase9:0 had [] on shutdown 2023-07-11 14:36:45,564 DEBUG [M:0;jenkins-hbase9:32785] master.HMaster(1512): Stopping service threads 2023-07-11 14:36:45,564 INFO [M:0;jenkins-hbase9:32785] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-11 14:36:45,565 ERROR [M:0;jenkins-hbase9:32785] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-11 14:36:45,565 INFO [M:0;jenkins-hbase9:32785] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-11 14:36:45,565 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-11 14:36:45,565 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-11 14:36:45,565 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:45,565 DEBUG [M:0;jenkins-hbase9:32785] zookeeper.ZKUtil(398): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-11 14:36:45,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:45,566 WARN [M:0;jenkins-hbase9:32785] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-11 14:36:45,566 INFO [M:0;jenkins-hbase9:32785] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-11 14:36:45,566 INFO [M:0;jenkins-hbase9:32785] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-11 14:36:45,567 DEBUG [M:0;jenkins-hbase9:32785] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-11 14:36:45,567 INFO [M:0;jenkins-hbase9:32785] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:45,567 DEBUG [M:0;jenkins-hbase9:32785] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:45,567 DEBUG [M:0;jenkins-hbase9:32785] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-11 14:36:45,567 DEBUG [M:0;jenkins-hbase9:32785] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:45,567 INFO [M:0;jenkins-hbase9:32785] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=36.38 KB heapSize=46.74 KB 2023-07-11 14:36:45,582 INFO [M:0;jenkins-hbase9:32785] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=36.38 KB at sequenceid=238 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/6a562af7ec00414c949a77a545783078 2023-07-11 14:36:45,587 INFO [M:0;jenkins-hbase9:32785] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6a562af7ec00414c949a77a545783078 2023-07-11 14:36:45,588 DEBUG [M:0;jenkins-hbase9:32785] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/6a562af7ec00414c949a77a545783078 as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6a562af7ec00414c949a77a545783078 2023-07-11 14:36:45,593 INFO [M:0;jenkins-hbase9:32785] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6a562af7ec00414c949a77a545783078 2023-07-11 14:36:45,593 INFO [M:0;jenkins-hbase9:32785] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6a562af7ec00414c949a77a545783078, entries=20, sequenceid=238, filesize=5.6 K 2023-07-11 14:36:45,594 INFO [M:0;jenkins-hbase9:32785] regionserver.HRegion(2948): Finished flush of dataSize ~36.38 KB/37256, heapSize ~46.73 KB/47848, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 27ms, sequenceid=238, compaction requested=false 2023-07-11 14:36:45,597 INFO [M:0;jenkins-hbase9:32785] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:45,597 DEBUG [M:0;jenkins-hbase9:32785] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:45,597 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,32785,1689086172324 2023-07-11 14:36:45,605 INFO [M:0;jenkins-hbase9:32785] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-11 14:36:45,605 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:45,606 INFO [M:0;jenkins-hbase9:32785] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:32785 2023-07-11 14:36:45,608 DEBUG [M:0;jenkins-hbase9:32785] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase9.apache.org,32785,1689086172324 already deleted, retry=false 2023-07-11 14:36:45,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:45,658 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:45,757 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:45,757 INFO [M:0;jenkins-hbase9:32785] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,32785,1689086172324; zookeeper connection closed. 2023-07-11 14:36:45,757 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:32785-0x10154c21b79001c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:45,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:45,763 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:45,857 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:45,857 INFO [RS:0;jenkins-hbase9:42891] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,42891,1689086172361; zookeeper connection closed. 2023-07-11 14:36:45,857 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:42891-0x10154c21b79001d, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:45,858 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7b991c42] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7b991c42 2023-07-11 14:36:45,858 INFO [Listener at localhost/42861] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-11 14:36:45,860 DEBUG [hconnection-0xfe1cae7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:45,863 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45402, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:45,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-11 14:36:45,869 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:45,977 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:45,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10009 2023-07-11 14:36:45,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 87 WAL entries eligible for replication 2023-07-11 14:36:45,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10009 2023-07-11 14:36:45,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/3bfd16063a0672430ddd816d03e73f73/112=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/113=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/114=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/115=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/116=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/117=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/118=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/119=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/120=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/121=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/122=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/123=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/124=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/125=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/126=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/127=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/128=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/129=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/130=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/131=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/132=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/133=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/134=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/135=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/136=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/137=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/138=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/139=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/140=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/141=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/142=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/143=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/144=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/145=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/146=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/147=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/148=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/149=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/150=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/151=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/152=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/153=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/154=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/155=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/156=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/157=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/158=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/159=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/160=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/161=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/162=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/163=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/164=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/165=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/166=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/167=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/168=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/169=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/170=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/171=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/172=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/173=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/174=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/175=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/176=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/177=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/178=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/179=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/180=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/181=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/182=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/183=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/184=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/185=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/186=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/187=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/188=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/189=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/190=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/191=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/192=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/193=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/194=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/195=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/196=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/197=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/198=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1.1689086204154, lastWalPosition=10009, nbRowKeys=87, nbHFiles=0, heapSize=18096, lastSeqIds={}, endOfFile=false,usedBufferSize=18096] 2023-07-11 14:36:45,985 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-11 14:36:46,083 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:46,086 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-11 14:36:46,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 19747 2023-07-11 14:36:46,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 91 WAL entries eligible for replication 2023-07-11 14:36:46,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 19747 2023-07-11 14:36:46,189 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-11 14:36:46,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 31907 2023-07-11 14:36:46,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 113 WAL entries eligible for replication 2023-07-11 14:36:46,220 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x11ec8647 to 127.0.0.1:64930 2023-07-11 14:36:46,220 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:46,220 DEBUG [Listener at localhost/42861] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-11 14:36:46,220 DEBUG [Listener at localhost/42861] util.JVMClusterUtil(257): Found active master hash=422974377, stopped=false 2023-07-11 14:36:46,220 INFO [Listener at localhost/42861] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:46,222 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-11 14:36:46,223 INFO [Listener at localhost/42861] procedure2.ProcedureExecutor(629): Stopping 2023-07-11 14:36:46,222 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-11 14:36:46,223 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:46,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-11 14:36:46,226 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x004b76e9 to 127.0.0.1:64930 2023-07-11 14:36:46,226 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:46,226 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-11 14:36:46,226 INFO [Listener at localhost/42861] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,44721,1689086199257' ***** 2023-07-11 14:36:46,226 INFO [Listener at localhost/42861] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-11 14:36:46,226 INFO [RS:0;jenkins-hbase9:44721] regionserver.HeapMemoryManager(220): Stopping 2023-07-11 14:36:46,226 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-11 14:36:46,226 INFO [RS:0;jenkins-hbase9:44721] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-11 14:36:46,227 INFO [RS:0;jenkins-hbase9:44721] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-11 14:36:46,227 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(3305): Received CLOSE for 11796b6d2ba24476446363b9a5701d72 2023-07-11 14:36:46,227 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(3305): Received CLOSE for 3bfd16063a0672430ddd816d03e73f73 2023-07-11 14:36:46,228 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(3305): Received CLOSE for ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:46,228 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:46,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 11796b6d2ba24476446363b9a5701d72, disabling compactions & flushes 2023-07-11 14:36:46,228 DEBUG [RS:0;jenkins-hbase9:44721] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7864cd48 to 127.0.0.1:64930 2023-07-11 14:36:46,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:46,228 DEBUG [RS:0;jenkins-hbase9:44721] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:46,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:46,228 INFO [RS:0;jenkins-hbase9:44721] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-11 14:36:46,229 INFO [RS:0;jenkins-hbase9:44721] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-11 14:36:46,229 INFO [RS:0;jenkins-hbase9:44721] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-11 14:36:46,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. after waiting 0 ms 2023-07-11 14:36:46,229 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-11 14:36:46,229 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:46,229 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-11 14:36:46,229 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1478): Online Regions={11796b6d2ba24476446363b9a5701d72=hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72., 1588230740=hbase:meta,,1.1588230740, 3bfd16063a0672430ddd816d03e73f73=t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73., ef703c6f28122a7c70d67fbbd278729c=t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c.} 2023-07-11 14:36:46,230 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-11 14:36:46,230 DEBUG [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1504): Waiting on 11796b6d2ba24476446363b9a5701d72, 1588230740, 3bfd16063a0672430ddd816d03e73f73, ef703c6f28122a7c70d67fbbd278729c 2023-07-11 14:36:46,230 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-11 14:36:46,230 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-11 14:36:46,230 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-11 14:36:46,230 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-11 14:36:46,230 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-07-11 14:36:46,245 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/namespace/11796b6d2ba24476446363b9a5701d72/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-11 14:36:46,247 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:46,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:46,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 11796b6d2ba24476446363b9a5701d72: 2023-07-11 14:36:46,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689086161642.11796b6d2ba24476446363b9a5701d72. 2023-07-11 14:36:46,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 3bfd16063a0672430ddd816d03e73f73, disabling compactions & flushes 2023-07-11 14:36:46,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:46,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:46,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. after waiting 0 ms 2023-07-11 14:36:46,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:46,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 3bfd16063a0672430ddd816d03e73f73 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-07-11 14:36:46,250 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/.tmp/info/f74fd206b4f1464ba56776e198afbac7 2023-07-11 14:36:46,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/.tmp/cf1/b1937d423b80494a90cbc848e3d1e226 2023-07-11 14:36:46,287 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-11 14:36:46,288 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/.tmp/rep_barrier/23aef8116e3c4e1a9965d4ebf2ff3fa7 2023-07-11 14:36:46,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/.tmp/norep/7d26c46b10104a14bc39ede3f3c97b10 2023-07-11 14:36:46,298 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-11 14:36:46,298 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/.tmp/info/f74fd206b4f1464ba56776e198afbac7 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/info/f74fd206b4f1464ba56776e198afbac7 2023-07-11 14:36:46,298 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-11 14:36:46,298 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-11 14:36:46,298 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0.1689086199666, lastWalPosition=728, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-11 14:36:46,307 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/.tmp/cf1/b1937d423b80494a90cbc848e3d1e226 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/cf1/b1937d423b80494a90cbc848e3d1e226 2023-07-11 14:36:46,310 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/info/f74fd206b4f1464ba56776e198afbac7, entries=24, sequenceid=31, filesize=7.4 K 2023-07-11 14:36:46,311 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/.tmp/rep_barrier/23aef8116e3c4e1a9965d4ebf2ff3fa7 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/rep_barrier/23aef8116e3c4e1a9965d4ebf2ff3fa7 2023-07-11 14:36:46,315 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/cf1/b1937d423b80494a90cbc848e3d1e226, entries=100, sequenceid=213, filesize=8.4 K 2023-07-11 14:36:46,318 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/.tmp/norep/7d26c46b10104a14bc39ede3f3c97b10 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/norep/7d26c46b10104a14bc39ede3f3c97b10 2023-07-11 14:36:46,320 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/rep_barrier/23aef8116e3c4e1a9965d4ebf2ff3fa7, entries=2, sequenceid=31, filesize=4.9 K 2023-07-11 14:36:46,321 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 91ms, sequenceid=31, compaction requested=false 2023-07-11 14:36:46,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/norep/7d26c46b10104a14bc39ede3f3c97b10, entries=1, sequenceid=213, filesize=4.8 K 2023-07-11 14:36:46,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for 3bfd16063a0672430ddd816d03e73f73 in 89ms, sequenceid=213, compaction requested=false 2023-07-11 14:36:46,356 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-11 14:36:46,357 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-11 14:36:46,357 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-11 14:36:46,357 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-11 14:36:46,357 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-11 14:36:46,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t1_syncup/3bfd16063a0672430ddd816d03e73f73/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-07-11 14:36:46,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:46,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 3bfd16063a0672430ddd816d03e73f73: 2023-07-11 14:36:46,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689086165412.3bfd16063a0672430ddd816d03e73f73. 2023-07-11 14:36:46,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing ef703c6f28122a7c70d67fbbd278729c, disabling compactions & flushes 2023-07-11 14:36:46,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:46,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:46,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. after waiting 0 ms 2023-07-11 14:36:46,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:46,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing ef703c6f28122a7c70d67fbbd278729c 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-07-11 14:36:46,372 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/.tmp/cf1/49360e7cf97f41e38517995ba074ba27 2023-07-11 14:36:46,388 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/.tmp/norep/1df95d83eefa4f049ffef3636e7713e1 2023-07-11 14:36:46,394 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/.tmp/cf1/49360e7cf97f41e38517995ba074ba27 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/cf1/49360e7cf97f41e38517995ba074ba27 2023-07-11 14:36:46,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/cf1/49360e7cf97f41e38517995ba074ba27, entries=200, sequenceid=413, filesize=12.5 K 2023-07-11 14:36:46,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/.tmp/norep/1df95d83eefa4f049ffef3636e7713e1 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/norep/1df95d83eefa4f049ffef3636e7713e1 2023-07-11 14:36:46,403 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-11 14:36:46,406 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/norep/1df95d83eefa4f049ffef3636e7713e1, entries=1, sequenceid=413, filesize=4.8 K 2023-07-11 14:36:46,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for ef703c6f28122a7c70d67fbbd278729c in 47ms, sequenceid=413, compaction requested=false 2023-07-11 14:36:46,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/data/default/t2_syncup/ef703c6f28122a7c70d67fbbd278729c/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-07-11 14:36:46,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:46,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for ef703c6f28122a7c70d67fbbd278729c: 2023-07-11 14:36:46,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689086166052.ef703c6f28122a7c70d67fbbd278729c. 2023-07-11 14:36:46,430 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,44721,1689086199257; all regions closed. 2023-07-11 14:36:46,430 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:46,436 DEBUG [RS:0;jenkins-hbase9:44721] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs 2023-07-11 14:36:46,436 INFO [RS:0;jenkins-hbase9:44721] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C44721%2C1689086199257.meta:.meta(num 1689086199767) 2023-07-11 14:36:46,437 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:46,442 DEBUG [RS:0;jenkins-hbase9:44721] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs 2023-07-11 14:36:46,442 INFO [RS:0;jenkins-hbase9:44721] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1:(num 1689086204154) 2023-07-11 14:36:46,442 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:46,448 DEBUG [RS:0;jenkins-hbase9:44721] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs 2023-07-11 14:36:46,448 INFO [RS:0;jenkins-hbase9:44721] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0:(num 1689086199666) 2023-07-11 14:36:46,448 DEBUG [RS:0;jenkins-hbase9:44721] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:46,448 INFO [RS:0;jenkins-hbase9:44721] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:36:46,448 INFO [RS:0;jenkins-hbase9:44721] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-11 14:36:46,448 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:46,448 INFO [RS:0;jenkins-hbase9:44721] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-11 14:36:46,505 INFO [RS:0;jenkins-hbase9:44721.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/WALs/jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0.1689086199666 was moved to hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0.1689086199666 2023-07-11 14:36:46,587 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-07-11 14:36:46,624 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b79002f, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:46,624 DEBUG [RS:0;jenkins-hbase9:44721] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5c214aec to 127.0.0.1:64930 2023-07-11 14:36:46,624 DEBUG [RS:0;jenkins-hbase9:44721.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b79002f, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:46,624 DEBUG [RS:0;jenkins-hbase9:44721] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:46,726 INFO [RS:0;jenkins-hbase9:44721] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1 terminated 2023-07-11 14:36:46,727 INFO [RS:0;jenkins-hbase9:44721] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase9:44721.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1 terminated 2023-07-11 14:36:46,728 INFO [RS:0;jenkins-hbase9:44721] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:44721 2023-07-11 14:36:46,738 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:46,738 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-11 14:36:46,738 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-11 14:36:46,739 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,44721,1689086199257] 2023-07-11 14:36:46,739 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,44721,1689086199257; numProcessing=1 2023-07-11 14:36:46,740 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase9.apache.org,44721,1689086199257 already deleted, retry=false 2023-07-11 14:36:46,740 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,44721,1689086199257 expired; onlineServers=0 2023-07-11 14:36:46,740 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,44621,1689086199226' ***** 2023-07-11 14:36:46,740 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-11 14:36:46,741 DEBUG [M:0;jenkins-hbase9:44621] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d099457, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:46,741 INFO [M:0;jenkins-hbase9:44621] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:46,741 INFO [M:0;jenkins-hbase9:44621] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,44621,1689086199226; all regions closed. 2023-07-11 14:36:46,741 DEBUG [M:0;jenkins-hbase9:44621] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:46,741 DEBUG [M:0;jenkins-hbase9:44621] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-11 14:36:46,741 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-11 14:36:46,741 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086199437] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086199437,5,FailOnTimeoutGroup] 2023-07-11 14:36:46,741 DEBUG [M:0;jenkins-hbase9:44621] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-11 14:36:46,741 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086199437] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086199437,5,FailOnTimeoutGroup] 2023-07-11 14:36:46,742 INFO [M:0;jenkins-hbase9:44621] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-11 14:36:46,743 INFO [M:0;jenkins-hbase9:44621] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-11 14:36:46,743 INFO [M:0;jenkins-hbase9:44621] hbase.ChoreService(369): Chore service for: master/jenkins-hbase9:0 had [] on shutdown 2023-07-11 14:36:46,743 DEBUG [M:0;jenkins-hbase9:44621] master.HMaster(1512): Stopping service threads 2023-07-11 14:36:46,743 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-11 14:36:46,743 INFO [M:0;jenkins-hbase9:44621] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-11 14:36:46,743 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-11 14:36:46,744 ERROR [M:0;jenkins-hbase9:44621] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-11 14:36:46,744 INFO [M:0;jenkins-hbase9:44621] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-11 14:36:46,744 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-11 14:36:46,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-11 14:36:46,744 DEBUG [M:0;jenkins-hbase9:44621] zookeeper.ZKUtil(398): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-11 14:36:46,744 WARN [M:0;jenkins-hbase9:44621] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-11 14:36:46,744 INFO [M:0;jenkins-hbase9:44621] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-11 14:36:46,745 INFO [M:0;jenkins-hbase9:44621] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-11 14:36:46,747 DEBUG [M:0;jenkins-hbase9:44621] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-11 14:36:46,747 INFO [M:0;jenkins-hbase9:44621] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:46,747 DEBUG [M:0;jenkins-hbase9:44621] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:46,747 DEBUG [M:0;jenkins-hbase9:44621] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-11 14:36:46,747 DEBUG [M:0;jenkins-hbase9:44621] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:46,747 INFO [M:0;jenkins-hbase9:44621] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.40 KB heapSize=36.57 KB 2023-07-11 14:36:46,762 INFO [M:0;jenkins-hbase9:44621] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.40 KB at sequenceid=217 (bloomFilter=true), to=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5faa7b7d236b463da4c5797af422a564 2023-07-11 14:36:46,767 DEBUG [M:0;jenkins-hbase9:44621] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5faa7b7d236b463da4c5797af422a564 as hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5faa7b7d236b463da4c5797af422a564 2023-07-11 14:36:46,772 INFO [M:0;jenkins-hbase9:44621] regionserver.HStore(1080): Added hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5faa7b7d236b463da4c5797af422a564, entries=10, sequenceid=217, filesize=5.8 K 2023-07-11 14:36:46,773 INFO [M:0;jenkins-hbase9:44621] regionserver.HRegion(2948): Finished flush of dataSize ~29.40 KB/30105, heapSize ~36.55 KB/37432, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 26ms, sequenceid=217, compaction requested=false 2023-07-11 14:36:46,776 INFO [M:0;jenkins-hbase9:44621] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:36:46,776 DEBUG [M:0;jenkins-hbase9:44621] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:46,777 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/MasterData/WALs/jenkins-hbase9.apache.org,44621,1689086199226 2023-07-11 14:36:46,785 INFO [M:0;jenkins-hbase9:44621] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-11 14:36:46,785 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:36:46,785 INFO [M:0;jenkins-hbase9:44621] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:44621 2023-07-11 14:36:46,788 DEBUG [M:0;jenkins-hbase9:44621] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase9.apache.org,44621,1689086199226 already deleted, retry=false 2023-07-11 14:36:46,839 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:46,839 INFO [RS:0;jenkins-hbase9:44721] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,44721,1689086199257; zookeeper connection closed. 2023-07-11 14:36:46,839 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:44721-0x10154c21b79002b, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:46,841 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@67728966] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@67728966 2023-07-11 14:36:46,841 INFO [Listener at localhost/42861] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-11 14:36:46,939 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:46,939 INFO [M:0;jenkins-hbase9:44621] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,44621,1689086199226; zookeeper connection closed. 2023-07-11 14:36:46,939 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:44621-0x10154c21b79002a, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:36:46,943 INFO [Listener at localhost/42861] client.ConnectionUtils(127): master/jenkins-hbase9:0 server-side Connection retries=18 2023-07-11 14:36:46,944 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:46,944 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:46,944 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:46,944 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:46,944 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:46,944 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:46,946 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:38725 2023-07-11 14:36:46,946 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:46,948 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:46,949 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38725 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:46,958 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:387250x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:46,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38725-0x10154c21b790032 connected 2023-07-11 14:36:46,961 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:46,962 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:46,962 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-11 14:36:46,963 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38725 2023-07-11 14:36:46,963 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38725 2023-07-11 14:36:46,964 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38725 2023-07-11 14:36:46,964 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38725 2023-07-11 14:36:46,964 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38725 2023-07-11 14:36:46,964 INFO [Listener at localhost/42861] master.HMaster(444): hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902, hbase.cluster.distributed=false 2023-07-11 14:36:46,965 DEBUG [pool-658-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-11 14:36:46,979 INFO [Listener at localhost/42861] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=18 2023-07-11 14:36:46,979 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:46,979 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:46,979 INFO [Listener at localhost/42861] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-11 14:36:46,979 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-11 14:36:46,979 INFO [Listener at localhost/42861] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-11 14:36:46,979 INFO [Listener at localhost/42861] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-11 14:36:46,984 INFO [Listener at localhost/42861] ipc.NettyRpcServer(120): Bind to /172.31.2.10:40099 2023-07-11 14:36:46,984 INFO [Listener at localhost/42861] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-11 14:36:46,988 DEBUG [Listener at localhost/42861] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-11 14:36:46,989 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:46,990 INFO [Listener at localhost/42861] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:46,992 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40099 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:46,994 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:400990x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:46,996 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:400990x0, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:36:46,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40099-0x10154c21b790033 connected 2023-07-11 14:36:46,997 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:36:46,997 DEBUG [Listener at localhost/42861] zookeeper.ZKUtil(164): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-11 14:36:46,999 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40099 2023-07-11 14:36:46,999 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40099 2023-07-11 14:36:47,000 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40099 2023-07-11 14:36:47,000 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40099 2023-07-11 14:36:47,000 DEBUG [Listener at localhost/42861] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40099 2023-07-11 14:36:47,002 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:36:47,003 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-11 14:36:47,003 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:36:47,011 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-11 14:36:47,011 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-11 14:36:47,011 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:47,011 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-11 14:36:47,013 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase9.apache.org,38725,1689086206943 from backup master directory 2023-07-11 14:36:47,013 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-11 14:36:47,014 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:36:47,014 WARN [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:47,014 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-11 14:36:47,014 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:36:47,026 INFO [master/jenkins-hbase9:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-11 14:36:47,071 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2490333e to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:47,077 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78231e2e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:47,077 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-11 14:36:47,077 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-11 14:36:47,078 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:47,079 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:47,084 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,32785,1689086172324 to hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,32785,1689086172324-dead as it is dead 2023-07-11 14:36:47,085 INFO [master/jenkins-hbase9:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,32785,1689086172324-dead/jenkins-hbase9.apache.org%2C32785%2C1689086172324.jenkins-hbase9.apache.org%2C32785%2C1689086172324.regiongroup-0.1689086172491 2023-07-11 14:36:47,085 INFO [master/jenkins-hbase9:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,32785,1689086172324-dead/jenkins-hbase9.apache.org%2C32785%2C1689086172324.jenkins-hbase9.apache.org%2C32785%2C1689086172324.regiongroup-0.1689086172491 after 0ms 2023-07-11 14:36:47,086 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,32785,1689086172324-dead/jenkins-hbase9.apache.org%2C32785%2C1689086172324.jenkins-hbase9.apache.org%2C32785%2C1689086172324.regiongroup-0.1689086172491 to hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C32785%2C1689086172324.jenkins-hbase9.apache.org%2C32785%2C1689086172324.regiongroup-0.1689086172491 2023-07-11 14:36:47,086 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,32785,1689086172324-dead 2023-07-11 14:36:47,086 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:36:47,087 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:47,088 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C38725%2C1689086206943.jenkins-hbase9.apache.org%2C38725%2C1689086206943.regiongroup-0, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,38725,1689086206943, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/oldWALs, maxLogs=10 2023-07-11 14:36:47,095 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,38725,1689086206943/jenkins-hbase9.apache.org%2C38725%2C1689086206943.jenkins-hbase9.apache.org%2C38725%2C1689086206943.regiongroup-0.1689086207088 2023-07-11 14:36:47,095 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK], DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK], DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK]] 2023-07-11 14:36:47,095 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:47,095 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:47,095 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:47,095 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:47,098 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:47,098 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-11 14:36:47,099 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-11 14:36:47,105 INFO [StoreFileOpener-proc-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6a562af7ec00414c949a77a545783078 2023-07-11 14:36:47,105 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6a562af7ec00414c949a77a545783078 2023-07-11 14:36:47,108 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bafe90e861e748fda9018cd963db289a 2023-07-11 14:36:47,108 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:47,109 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-11 14:36:47,109 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C32785%2C1689086172324.jenkins-hbase9.apache.org%2C32785%2C1689086172324.regiongroup-0.1689086172491 2023-07-11 14:36:47,112 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 139, firstSequenceIdInLog=120, maxSequenceIdInLog=240, path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C32785%2C1689086172324.jenkins-hbase9.apache.org%2C32785%2C1689086172324.regiongroup-0.1689086172491 2023-07-11 14:36:47,113 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase9.apache.org%2C32785%2C1689086172324.jenkins-hbase9.apache.org%2C32785%2C1689086172324.regiongroup-0.1689086172491 2023-07-11 14:36:47,115 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-11 14:36:47,117 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/240.seqid, newMaxSeqId=240, maxSeqId=118 2023-07-11 14:36:47,118 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=241; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60335557, jitterRate=-0.10093013942241669}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-11 14:36:47,118 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:36:47,118 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-11 14:36:47,119 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-11 14:36:47,119 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-11 14:36:47,119 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-11 14:36:47,120 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-11 14:36:47,125 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-11 14:36:47,126 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-11 14:36:47,126 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-11 14:36:47,126 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-11 14:36:47,127 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-11 14:36:47,127 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-11 14:36:47,127 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-11 14:36:47,127 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-11 14:36:47,128 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-11 14:36:47,129 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,42891,1689086172361, table=hbase:meta, region=1588230740 2023-07-11 14:36:47,130 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-07-11 14:36:47,132 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,42891,1689086172361 already deleted, retry=false 2023-07-11 14:36:47,132 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase9.apache.org,42891,1689086172361 on jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:36:47,135 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase9.apache.org,42891,1689086172361, splitWal=true, meta=true 2023-07-11 14:36:47,135 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase9.apache.org,42891,1689086172361 (carryingMeta=true) jenkins-hbase9.apache.org,42891,1689086172361/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@e99ea60[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-11 14:36:47,145 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-11 14:36:47,145 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-11 14:36:47,146 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-11 14:36:47,146 INFO [master/jenkins-hbase9:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-11 14:36:47,146 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-11 14:36:47,147 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-11 14:36:47,147 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-11 14:36:47,148 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-11 14:36:47,149 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-11 14:36:47,150 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-11 14:36:47,150 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:36:47,150 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase9.apache.org,38725,1689086206943, sessionid=0x10154c21b790032, setting cluster-up flag (Was=false) 2023-07-11 14:36:47,154 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-11 14:36:47,155 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:36:47,159 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-11 14:36:47,159 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:36:47,160 WARN [master/jenkins-hbase9:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/.hbase-snapshot/.tmp 2023-07-11 14:36:47,161 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:47,161 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:47,161 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:47,161 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-07-11 14:36:47,161 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase9:0, corePoolSize=10, maxPoolSize=10 2023-07-11 14:36:47,161 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,161 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:47,161 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,167 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689086237167 2023-07-11 14:36:47,168 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-11 14:36:47,168 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-11 14:36:47,168 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-11 14:36:47,168 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-11 14:36:47,168 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-11 14:36:47,168 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-11 14:36:47,168 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:47,169 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-11 14:36:47,169 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-11 14:36:47,170 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase9.apache.org,42891,1689086172361; numProcessing=1 2023-07-11 14:36:47,170 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-11 14:36:47,170 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,42891,1689086172361, splitWal=true, meta=true 2023-07-11 14:36:47,170 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-11 14:36:47,170 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-11 14:36:47,171 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,42891,1689086172361, splitWal=true, meta=true, isMeta: true 2023-07-11 14:36:47,173 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086207170,5,FailOnTimeoutGroup] 2023-07-11 14:36:47,173 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086207173,5,FailOnTimeoutGroup] 2023-07-11 14:36:47,174 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:47,174 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-11 14:36:47,174 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:47,174 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:47,174 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689086207174, completionTime=-1 2023-07-11 14:36:47,174 WARN [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-11 14:36:47,174 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-11 14:36:47,174 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361-splitting 2023-07-11 14:36:47,175 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361-splitting dir is empty, no logs to split. 2023-07-11 14:36:47,175 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase9.apache.org,42891,1689086172361 WAL count=0, meta=true 2023-07-11 14:36:47,177 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361-splitting dir is empty, no logs to split. 2023-07-11 14:36:47,178 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase9.apache.org,42891,1689086172361 WAL count=0, meta=true 2023-07-11 14:36:47,178 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase9.apache.org,42891,1689086172361 WAL splitting is done? wals=0, meta=true 2023-07-11 14:36:47,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-11 14:36:47,181 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-11 14:36:47,182 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:47,202 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(951): ClusterId : 6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:47,202 DEBUG [RS:0;jenkins-hbase9:40099] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-11 14:36:47,204 DEBUG [RS:0;jenkins-hbase9:40099] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-11 14:36:47,204 DEBUG [RS:0;jenkins-hbase9:40099] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-11 14:36:47,205 DEBUG [RS:0;jenkins-hbase9:40099] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-11 14:36:47,207 DEBUG [RS:0;jenkins-hbase9:40099] zookeeper.ReadOnlyZKClient(139): Connect 0x3a739fd8 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:47,211 DEBUG [RS:0;jenkins-hbase9:40099] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@34b466e6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:47,211 DEBUG [RS:0;jenkins-hbase9:40099] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@308bf4a8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:36:47,219 DEBUG [RS:0;jenkins-hbase9:40099] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase9:40099 2023-07-11 14:36:47,219 INFO [RS:0;jenkins-hbase9:40099] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-11 14:36:47,219 INFO [RS:0;jenkins-hbase9:40099] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-11 14:36:47,219 DEBUG [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1022): About to register with Master. 2023-07-11 14:36:47,220 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase9.apache.org,38725,1689086206943 with isa=jenkins-hbase9.apache.org/172.31.2.10:40099, startcode=1689086206978 2023-07-11 14:36:47,220 DEBUG [RS:0;jenkins-hbase9:40099] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-11 14:36:47,223 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:52313, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-11 14:36:47,224 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38725] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:47,224 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-11 14:36:47,224 DEBUG [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902 2023-07-11 14:36:47,224 DEBUG [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33985 2023-07-11 14:36:47,224 DEBUG [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-11 14:36:47,227 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:47,227 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b790026, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:36:47,227 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-11 14:36:47,227 DEBUG [RS:0;jenkins-hbase9:40099] zookeeper.ZKUtil(162): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:47,228 WARN [RS:0;jenkins-hbase9:40099] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-11 14:36:47,228 INFO [RS:0;jenkins-hbase9:40099] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:47,228 INFO [RS:0;jenkins-hbase9:40099] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:47,228 DEBUG [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:47,229 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,40099,1689086206978] 2023-07-11 14:36:47,233 DEBUG [RS:0;jenkins-hbase9:40099] zookeeper.ZKUtil(162): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:47,233 DEBUG [RS:0;jenkins-hbase9:40099] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:47,234 INFO [RS:0;jenkins-hbase9:40099] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-11 14:36:47,235 INFO [RS:0;jenkins-hbase9:40099] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-11 14:36:47,235 INFO [RS:0;jenkins-hbase9:40099] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-11 14:36:47,236 INFO [RS:0;jenkins-hbase9:40099] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:47,239 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-11 14:36:47,240 INFO [RS:0;jenkins-hbase9:40099] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:47,241 DEBUG [RS:0;jenkins-hbase9:40099] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,241 DEBUG [RS:0;jenkins-hbase9:40099] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,241 DEBUG [RS:0;jenkins-hbase9:40099] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,241 DEBUG [RS:0;jenkins-hbase9:40099] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,241 DEBUG [RS:0;jenkins-hbase9:40099] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,241 DEBUG [RS:0;jenkins-hbase9:40099] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-07-11 14:36:47,241 DEBUG [RS:0;jenkins-hbase9:40099] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,241 DEBUG [RS:0;jenkins-hbase9:40099] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,241 DEBUG [RS:0;jenkins-hbase9:40099] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,241 DEBUG [RS:0;jenkins-hbase9:40099] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-07-11 14:36:47,242 INFO [RS:0;jenkins-hbase9:40099] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:47,242 INFO [RS:0;jenkins-hbase9:40099] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:47,243 INFO [RS:0;jenkins-hbase9:40099] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:47,254 INFO [RS:0;jenkins-hbase9:40099] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-11 14:36:47,254 INFO [RS:0;jenkins-hbase9:40099] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,40099,1689086206978-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:47,264 INFO [RS:0;jenkins-hbase9:40099] regionserver.Replication(203): jenkins-hbase9.apache.org,40099,1689086206978 started 2023-07-11 14:36:47,264 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1637): Serving as jenkins-hbase9.apache.org,40099,1689086206978, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:40099, sessionid=0x10154c21b790033 2023-07-11 14:36:47,264 DEBUG [RS:0;jenkins-hbase9:40099] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-11 14:36:47,264 DEBUG [RS:0;jenkins-hbase9:40099] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:47,264 DEBUG [RS:0;jenkins-hbase9:40099] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,40099,1689086206978' 2023-07-11 14:36:47,264 DEBUG [RS:0;jenkins-hbase9:40099] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-11 14:36:47,265 DEBUG [RS:0;jenkins-hbase9:40099] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-11 14:36:47,265 DEBUG [RS:0;jenkins-hbase9:40099] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-11 14:36:47,265 DEBUG [RS:0;jenkins-hbase9:40099] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-11 14:36:47,265 DEBUG [RS:0;jenkins-hbase9:40099] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:47,265 DEBUG [RS:0;jenkins-hbase9:40099] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,40099,1689086206978' 2023-07-11 14:36:47,265 DEBUG [RS:0;jenkins-hbase9:40099] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-11 14:36:47,266 DEBUG [RS:0;jenkins-hbase9:40099] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-11 14:36:47,266 DEBUG [RS:0;jenkins-hbase9:40099] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-11 14:36:47,266 INFO [RS:0;jenkins-hbase9:40099] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-11 14:36:47,266 INFO [RS:0;jenkins-hbase9:40099] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-11 14:36:47,332 DEBUG [jenkins-hbase9:38725] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-11 14:36:47,334 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,40099,1689086206978, state=OPENING 2023-07-11 14:36:47,335 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-11 14:36:47,335 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-11 14:36:47,335 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,40099,1689086206978}] 2023-07-11 14:36:47,366 INFO [RS:0;jenkins-hbase9:40099] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:47,368 INFO [RS:0;jenkins-hbase9:40099] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C40099%2C1689086206978.jenkins-hbase9.apache.org%2C40099%2C1689086206978.regiongroup-0, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,40099,1689086206978, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:47,385 INFO [RS:0;jenkins-hbase9:40099] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,40099,1689086206978/jenkins-hbase9.apache.org%2C40099%2C1689086206978.jenkins-hbase9.apache.org%2C40099%2C1689086206978.regiongroup-0.1689086207369 2023-07-11 14:36:47,385 DEBUG [RS:0;jenkins-hbase9:40099] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK], DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK], DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK]] 2023-07-11 14:36:47,492 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:47,492 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:47,494 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55270, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-11 14:36:47,498 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-11 14:36:47,498 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:47,498 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:47,498 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:47,499 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C40099%2C1689086206978.meta, suffix=.meta, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,40099,1689086206978, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:47,506 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,40099,1689086206978/jenkins-hbase9.apache.org%2C40099%2C1689086206978.meta.1689086207500.meta 2023-07-11 14:36:47,506 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK], DatanodeInfoWithStorage[127.0.0.1:45009,DS-16f2c2d4-a880-4afb-a0b7-ea6e1caa6b73,DISK], DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK]] 2023-07-11 14:36:47,506 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:47,506 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-11 14:36:47,506 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-11 14:36:47,506 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-11 14:36:47,506 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-11 14:36:47,506 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:47,506 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-11 14:36:47,507 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-11 14:36:47,507 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-11 14:36:47,508 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info 2023-07-11 14:36:47,508 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info 2023-07-11 14:36:47,509 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-11 14:36:47,514 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info/1041948364674772bb741eb2149d53de 2023-07-11 14:36:47,518 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info/2023d570cfaa402593275b0451eb9042 2023-07-11 14:36:47,518 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:47,518 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-11 14:36:47,519 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:47,519 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/rep_barrier 2023-07-11 14:36:47,519 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-11 14:36:47,520 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:47,520 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-11 14:36:47,521 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table 2023-07-11 14:36:47,521 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table 2023-07-11 14:36:47,521 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-11 14:36:47,526 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/table/ed3b6a6f597941748150410cb9e7546c 2023-07-11 14:36:47,527 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:47,527 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740 2023-07-11 14:36:47,528 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740 2023-07-11 14:36:47,530 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-11 14:36:47,531 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-11 14:36:47,532 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66093150, jitterRate=-0.015135318040847778}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-11 14:36:47,532 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-11 14:36:47,534 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1689086207492 2023-07-11 14:36:47,538 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-11 14:36:47,539 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-11 14:36:47,539 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,40099,1689086206978, state=OPEN 2023-07-11 14:36:47,540 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-11 14:36:47,541 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-11 14:36:47,543 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-11 14:36:47,543 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,40099,1689086206978 in 206 msec 2023-07-11 14:36:47,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-07-11 14:36:47,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 364 msec 2023-07-11 14:36:48,726 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1552ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-11 14:36:49,660 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-11 14:36:50,229 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-11 14:36:51,681 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-11 14:36:51,681 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-11 14:36:51,684 DEBUG [hconnection-0x2d1f295c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:51,686 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45172, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:51,693 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=d4185bdf44467f073faaa5de49a197d7, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,42891,1689086172361, regionLocation=jenkins-hbase9.apache.org,42891,1689086172361, openSeqNum=10 2023-07-11 14:36:51,693 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=f27301259d11533bab9412d981c5782d, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,42891,1689086172361, regionLocation=jenkins-hbase9.apache.org,42891,1689086172361, openSeqNum=11 2023-07-11 14:36:51,693 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=34f512aa6fa4c9880527ac501e69b8b2, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,42891,1689086172361, regionLocation=jenkins-hbase9.apache.org,42891,1689086172361, openSeqNum=12 2023-07-11 14:36:51,693 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-11 14:36:51,693 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689086271693 2023-07-11 14:36:51,693 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689086331693 2023-07-11 14:36:51,693 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 12 msec 2023-07-11 14:36:51,709 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase9.apache.org,42891,1689086172361 had 4 regions 2023-07-11 14:36:51,709 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,38725,1689086206943-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:51,709 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,38725,1689086206943-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:51,709 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,38725,1689086206943-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:51,710 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase9:38725, period=300000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:51,710 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-11 14:36:51,710 WARN [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. is NOT online; state={d4185bdf44467f073faaa5de49a197d7 state=OPEN, ts=1689086211693, server=jenkins-hbase9.apache.org,42891,1689086172361}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-11 14:36:51,711 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase9.apache.org,42891,1689086172361, splitWal=true, meta=true, isMeta: false 2023-07-11 14:36:51,714 WARN [master/jenkins-hbase9:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase9.apache.org,42891,1689086172361/hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7., unknown_server=jenkins-hbase9.apache.org,42891,1689086172361/t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d., unknown_server=jenkins-hbase9.apache.org,42891,1689086172361/t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:51,714 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361-splitting dir is empty, no logs to split. 2023-07-11 14:36:51,714 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase9.apache.org,42891,1689086172361 WAL count=0, meta=false 2023-07-11 14:36:51,716 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,42891,1689086172361-splitting dir is empty, no logs to split. 2023-07-11 14:36:51,716 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase9.apache.org,42891,1689086172361 WAL count=0, meta=false 2023-07-11 14:36:51,716 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase9.apache.org,42891,1689086172361 WAL splitting is done? wals=0, meta=false 2023-07-11 14:36:51,720 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN}] 2023-07-11 14:36:51,721 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN 2023-07-11 14:36:51,721 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN 2023-07-11 14:36:51,721 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN 2023-07-11 14:36:51,722 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:51,722 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:51,722 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-11 14:36:51,872 DEBUG [jenkins-hbase9:38725] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-11 14:36:51,876 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=d4185bdf44467f073faaa5de49a197d7, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:51,876 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=34f512aa6fa4c9880527ac501e69b8b2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:51,876 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=f27301259d11533bab9412d981c5782d, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:51,876 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086211876"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086211876"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086211876"}]},"ts":"1689086211876"} 2023-07-11 14:36:51,876 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086211876"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086211876"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086211876"}]},"ts":"1689086211876"} 2023-07-11 14:36:51,876 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086211876"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689086211876"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689086211876"}]},"ts":"1689086211876"} 2023-07-11 14:36:51,879 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE; OpenRegionProcedure d4185bdf44467f073faaa5de49a197d7, server=jenkins-hbase9.apache.org,40099,1689086206978}] 2023-07-11 14:36:51,879 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 34f512aa6fa4c9880527ac501e69b8b2, server=jenkins-hbase9.apache.org,40099,1689086206978}] 2023-07-11 14:36:51,882 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=31, state=RUNNABLE; OpenRegionProcedure f27301259d11533bab9412d981c5782d, server=jenkins-hbase9.apache.org,40099,1689086206978}] 2023-07-11 14:36:52,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:52,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-11 14:36:52,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C40099%2C1689086206978.jenkins-hbase9.apache.org%2C40099%2C1689086206978.regiongroup-1, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,40099,1689086206978, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs, maxLogs=10 2023-07-11 14:36:52,043 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:52,043 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d4185bdf44467f073faaa5de49a197d7, NAME => 'hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:52,044 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:52,044 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:52,044 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:52,044 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:52,045 INFO [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:52,046 DEBUG [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/info 2023-07-11 14:36:52,046 DEBUG [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/info 2023-07-11 14:36:52,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,40099,1689086206978/jenkins-hbase9.apache.org%2C40099%2C1689086206978.jenkins-hbase9.apache.org%2C40099%2C1689086206978.regiongroup-1.1689086212039 2023-07-11 14:36:52,048 INFO [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4185bdf44467f073faaa5de49a197d7 columnFamilyName info 2023-07-11 14:36:52,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34139,DS-e758f896-76b0-41e4-9477-db5ffceae74c,DISK], DatanodeInfoWithStorage[127.0.0.1:40751,DS-8f4e7cfc-bacb-4f4b-b056-3150dd58f95c,DISK], DatanodeInfoWithStorage[127.0.0.1:38067,DS-b8f9d8d5-36e0-4e0b-835a-da34176c3eda,DISK]] 2023-07-11 14:36:52,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 34f512aa6fa4c9880527ac501e69b8b2, NAME => 't2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:52,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:52,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:52,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:52,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:52,049 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:52,050 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1 2023-07-11 14:36:52,050 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1 2023-07-11 14:36:52,050 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 34f512aa6fa4c9880527ac501e69b8b2 columnFamilyName cf1 2023-07-11 14:36:52,053 DEBUG [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/info/ca4f495a328442daad2ea4bf64ee841b 2023-07-11 14:36:52,053 INFO [StoreOpener-d4185bdf44467f073faaa5de49a197d7-1] regionserver.HStore(310): Store=d4185bdf44467f073faaa5de49a197d7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:52,054 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:52,055 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:52,057 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3976bd393a1a41dc89cbea674d1145b4 2023-07-11 14:36:52,057 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1/3976bd393a1a41dc89cbea674d1145b4 2023-07-11 14:36:52,058 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:36:52,059 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened d4185bdf44467f073faaa5de49a197d7; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66851592, jitterRate=-0.0038336515426635742}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-11 14:36:52,059 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for d4185bdf44467f073faaa5de49a197d7: 2023-07-11 14:36:52,061 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7., pid=33, masterSystemTime=1689086212034 2023-07-11 14:36:52,063 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1/e3e0f680c76e48bf80b60121dfd11e86 2023-07-11 14:36:52,063 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(310): Store=34f512aa6fa4c9880527ac501e69b8b2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:52,063 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:52,064 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:52,064 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:36:52,064 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=d4185bdf44467f073faaa5de49a197d7, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:52,064 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/norep 2023-07-11 14:36:52,065 DEBUG [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/norep 2023-07-11 14:36:52,065 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689086212064"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086212064"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086212064"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086212064"}]},"ts":"1689086212064"} 2023-07-11 14:36:52,065 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 34f512aa6fa4c9880527ac501e69b8b2 columnFamilyName norep 2023-07-11 14:36:52,065 INFO [StoreOpener-34f512aa6fa4c9880527ac501e69b8b2-1] regionserver.HStore(310): Store=34f512aa6fa4c9880527ac501e69b8b2/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:52,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:52,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:52,069 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=30 2023-07-11 14:36:52,069 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=30, state=SUCCESS; OpenRegionProcedure d4185bdf44467f073faaa5de49a197d7, server=jenkins-hbase9.apache.org,40099,1689086206978 in 187 msec 2023-07-11 14:36:52,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-11 14:36:52,070 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=d4185bdf44467f073faaa5de49a197d7, ASSIGN in 351 msec 2023-07-11 14:36:52,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:36:52,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 34f512aa6fa4c9880527ac501e69b8b2; next sequenceid=19; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72984424, jitterRate=0.08755266666412354}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-11 14:36:52,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 34f512aa6fa4c9880527ac501e69b8b2: 2023-07-11 14:36:52,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2., pid=34, masterSystemTime=1689086212034 2023-07-11 14:36:52,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:52,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:36:52,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:52,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f27301259d11533bab9412d981c5782d, NAME => 't1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.', STARTKEY => '', ENDKEY => ''} 2023-07-11 14:36:52,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup f27301259d11533bab9412d981c5782d 2023-07-11 14:36:52,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-11 14:36:52,075 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=34f512aa6fa4c9880527ac501e69b8b2, regionState=OPEN, openSeqNum=19, regionLocation=jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:52,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:52,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:52,076 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086212075"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086212075"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086212075"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086212075"}]},"ts":"1689086212075"} 2023-07-11 14:36:52,076 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f27301259d11533bab9412d981c5782d 2023-07-11 14:36:52,077 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1 2023-07-11 14:36:52,077 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1 2023-07-11 14:36:52,078 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f27301259d11533bab9412d981c5782d columnFamilyName cf1 2023-07-11 14:36:52,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-07-11 14:36:52,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 34f512aa6fa4c9880527ac501e69b8b2, server=jenkins-hbase9.apache.org,40099,1689086206978 in 198 msec 2023-07-11 14:36:52,081 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=34f512aa6fa4c9880527ac501e69b8b2, ASSIGN in 361 msec 2023-07-11 14:36:52,083 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1/37e3e6e8b34041bd8bc6a3e3df3c42bd 2023-07-11 14:36:52,087 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 969a0b4d2c414fd78805717f8ef78708 2023-07-11 14:36:52,087 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1/969a0b4d2c414fd78805717f8ef78708 2023-07-11 14:36:52,087 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(310): Store=f27301259d11533bab9412d981c5782d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:52,087 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region f27301259d11533bab9412d981c5782d 2023-07-11 14:36:52,088 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/norep 2023-07-11 14:36:52,088 DEBUG [StoreOpener-f27301259d11533bab9412d981c5782d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/norep 2023-07-11 14:36:52,089 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f27301259d11533bab9412d981c5782d columnFamilyName norep 2023-07-11 14:36:52,089 INFO [StoreOpener-f27301259d11533bab9412d981c5782d-1] regionserver.HStore(310): Store=f27301259d11533bab9412d981c5782d/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-11 14:36:52,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d 2023-07-11 14:36:52,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d 2023-07-11 14:36:52,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-11 14:36:52,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for f27301259d11533bab9412d981c5782d 2023-07-11 14:36:52,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened f27301259d11533bab9412d981c5782d; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70914065, jitterRate=0.056701913475990295}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-11 14:36:52,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for f27301259d11533bab9412d981c5782d: 2023-07-11 14:36:52,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d., pid=35, masterSystemTime=1689086212034 2023-07-11 14:36:52,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:52,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:36:52,097 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=f27301259d11533bab9412d981c5782d, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:36:52,097 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689086212097"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689086212097"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689086212097"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689086212097"}]},"ts":"1689086212097"} 2023-07-11 14:36:52,101 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=31 2023-07-11 14:36:52,101 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=31, state=SUCCESS; OpenRegionProcedure f27301259d11533bab9412d981c5782d, server=jenkins-hbase9.apache.org,40099,1689086206978 in 219 msec 2023-07-11 14:36:52,103 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=27 2023-07-11 14:36:52,103 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase9.apache.org,42891,1689086172361 after splitting done 2023-07-11 14:36:52,103 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=f27301259d11533bab9412d981c5782d, ASSIGN in 383 msec 2023-07-11 14:36:52,103 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase9.apache.org,42891,1689086172361 from processing; numProcessing=0 2023-07-11 14:36:52,105 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase9.apache.org,42891,1689086172361, splitWal=true, meta=true in 4.9710 sec 2023-07-11 14:36:52,264 INFO [jenkins-hbase9:40099Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:52,710 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-11 14:36:52,724 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-11 14:36:52,727 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-11 14:36:52,727 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.713sec 2023-07-11 14:36:52,727 INFO [master/jenkins-hbase9:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-11 14:36:52,727 INFO [master/jenkins-hbase9:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-11 14:36:52,727 INFO [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-11 14:36:52,727 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,38725,1689086206943-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-11 14:36:52,727 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,38725,1689086206943-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-11 14:36:52,730 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-11 14:36:52,806 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(139): Connect 0x225de9b7 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:52,810 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1df806db, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:52,811 DEBUG [hconnection-0x23b51306-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:52,813 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45188, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:52,815 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-11 14:36:52,816 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x225de9b7 to 127.0.0.1:64930 2023-07-11 14:36:52,816 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:36:52,817 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase9.apache.org:38725 after: jenkins-hbase9.apache.org:38725 2023-07-11 14:36:52,817 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(139): Connect 0x69c030b6 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:52,821 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@62e633ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:53,234 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-11 14:36:53,234 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-11 14:36:53,235 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-11 14:36:53,235 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-11 14:36:53,325 DEBUG [hconnection-0x6825fe11-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:53,328 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45190, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-11 14:36:53,348 INFO [Listener at localhost/42861] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689086213347 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:53,350 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication16890862133470x0, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:53,353 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689086213347-0x10154c21b790038 connected Start Replication Server start 2023-07-11 14:36:53,354 INFO [Listener at localhost/42861] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-11 14:36:53,355 INFO [Listener at localhost/42861] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-11 14:36:53,367 INFO [Listener at localhost/42861] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-11 14:36:53,368 DEBUG [Listener at localhost/42861] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-11 14:36:53,378 INFO [Listener at localhost/42861] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-11 14:36:53,379 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase9.apache.org,44721,1689086199257] other RSs: [] 2023-07-11 14:36:53,398 DEBUG [Listener at localhost/42861.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x2c2ae33d to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:53,402 DEBUG [Listener at localhost/42861.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@129967b1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:53,402 INFO [Listener at localhost/42861.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:53,405 DEBUG [Listener at localhost/42861.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:53,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10154c21b79003a connected 2023-07-11 14:36:53,407 INFO [Listener at localhost/42861.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 to cluster=6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:57,210 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase9.apache.org,44721,1689086199257/1's WALs to 1689086213354.syncuptool.replication.org,1234,1 2023-07-11 14:36:57,211 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1.1689086204154 with data PBUF\x08\xB7\x06 2023-07-11 14:36:57,212 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0.1689086199666 with data PBUF\x08\xD8\x05 2023-07-11 14:36:57,216 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase9.apache.org,44721,1689086199257/1's WALs to 1689086213354.syncuptool.replication.org,1234,1 2023-07-11 14:36:57,264 INFO [jenkins-hbase9:40099Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:36:57,547 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-11 14:36:59,218 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,44721,1689086199257] 2023-07-11 14:36:59,218 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,44721,1689086199257] 2023-07-11 14:36:59,228 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,44721,1689086199257] 2023-07-11 14:36:59,229 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase9.apache.org,44721,1689086199257, ReplicationSource: 1, currentBandwidth=0 2023-07-11 14:36:59,229 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:59,248 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257] zookeeper.ReadOnlyZKClient(139): Connect 0x14446692 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:59,253 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@74b98b7f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:59,253 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64930 2023-07-11 14:36:59,255 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-11 14:36:59,256 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10154c21b79003c connected 2023-07-11 14:36:59,256 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase9.apache.org,44721,1689086199257 (queues=2) is replicating from cluster=fbb5de0d-2980-4bc6-8c0e-9e3b8c604dd2 to cluster=6653eb33-b917-4e4f-b6bd-b9b511f5c1e6 2023-07-11 14:36:59,256 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1 2023-07-11 14:36:59,259 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase9.apache.org,44721,1689086199257, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-11 14:36:59,260 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0 2023-07-11 14:36:59,260 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1 2023-07-11 14:36:59,261 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase9.apache.org,44721,1689086199257, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-11 14:36:59,262 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0 2023-07-11 14:36:59,266 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0.1689086199666 2023-07-11 14:36:59,266 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-11 14:36:59,266 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase9.apache.org,44721,1689086199257 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-11 14:36:59,267 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0 of peer 1-jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:59,271 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1.1689086204154 2023-07-11 14:36:59,271 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceWALReader(162): Read 300 WAL entries eligible for replication 2023-07-11 14:36:59,271 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.wal-reader.jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-11 14:36:59,273 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase9.apache.org,44721,1689086199257 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/3bfd16063a0672430ddd816d03e73f73/112=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/113=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/114=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/115=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/116=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/117=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/118=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/119=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/120=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/121=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/122=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/123=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/124=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/125=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/126=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/127=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/128=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/129=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/130=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/131=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/132=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/133=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/134=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/135=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/136=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/137=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/138=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/139=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/140=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/141=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/142=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/143=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/144=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/145=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/146=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/147=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/148=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/149=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/150=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/151=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/152=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/153=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/154=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/155=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/156=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/157=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/158=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/159=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/160=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/161=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/162=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/163=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/164=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/165=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/166=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/167=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/168=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/169=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/170=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/171=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/172=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/173=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/174=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/175=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/176=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/177=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/178=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/179=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/180=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/181=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/182=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/183=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/184=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/185=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/186=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/187=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/188=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/189=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/190=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/191=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/192=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/193=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/194=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/195=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/196=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/197=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/198=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/199=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/200=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/201=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/202=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/203=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/204=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/205=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/206=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/207=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/208=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/209=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/210=[#edits: 1 = ],208}, {t1_syncup/3bfd16063a0672430ddd816d03e73f73/211=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/212=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/213=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/214=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/215=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/216=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/217=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/218=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/219=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/220=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/221=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/222=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/223=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/224=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/225=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/226=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/227=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/228=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/229=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/230=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/231=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/232=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/233=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/234=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/235=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/236=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/237=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/238=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/239=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/240=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/241=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/242=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/243=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/244=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/245=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/246=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/247=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/248=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/249=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/250=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/251=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/252=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/253=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/254=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/255=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/256=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/257=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/258=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/259=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/260=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/261=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/262=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/263=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/264=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/265=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/266=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/267=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/268=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/269=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/270=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/271=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/272=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/273=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/274=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/275=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/276=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/277=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/278=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/279=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/280=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/281=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/282=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/283=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/284=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/285=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/286=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/287=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/288=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/289=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/290=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/291=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/292=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/293=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/294=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/295=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/296=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/297=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/298=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/299=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/300=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/301=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/302=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/303=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/304=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/305=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/306=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/307=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/308=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/309=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/310=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/311=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/312=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/313=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/314=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/315=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/316=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/317=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/318=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/319=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/320=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/321=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/322=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/323=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/324=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/325=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/326=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/327=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/328=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/329=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/330=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/331=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/332=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/333=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/334=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/335=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/336=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/337=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/338=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/339=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/340=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/341=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/342=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/343=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/344=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/345=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/346=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/347=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/348=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/349=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/350=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/351=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/352=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/353=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/354=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/355=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/356=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/357=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/358=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/359=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/360=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/361=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/362=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/363=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/364=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/365=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/366=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/367=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/368=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/369=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/370=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/371=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/372=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/373=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/374=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/375=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/376=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/377=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/378=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/379=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/380=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/381=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/382=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/383=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/384=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/385=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/386=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/387=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/388=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/389=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/390=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/391=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/392=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/393=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/394=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/395=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/396=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/397=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/398=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/399=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/400=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/401=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/402=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/403=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/404=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/405=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/406=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/407=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/408=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/409=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/410=[#edits: 1 = ],208}, {t2_syncup/ef703c6f28122a7c70d67fbbd278729c/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41201/user/jenkins/test-data/169008fc-b6e4-348d-0e97-ff004d8cb77d/oldWALs/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1.1689086204154, lastWalPosition=34707, nbRowKeys=300, nbHFiles=0, heapSize=62400, lastSeqIds={}, endOfFile=true,usedBufferSize=62400] 2023-07-11 14:36:59,277 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-11 14:36:59,281 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45206, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-11 14:36:59,284 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40099] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-11 14:36:59,284 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40099] zookeeper.ReadOnlyZKClient(139): Connect 0x4ac85646 to 127.0.0.1:64930 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-11 14:36:59,289 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40099] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@440fb16f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-11 14:36:59,290 DEBUG [hconnection-0x40303d36-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-11 14:36:59,293 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45208, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-07-11 14:36:59,313 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40099] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-11 14:36:59,313 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase9.apache.org,44721,1689086199257] 2023-07-11 14:36:59,315 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1.1689086204154] 2023-07-11 14:36:59,317 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase9.apache.org,44721,1689086199257 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-11 14:36:59,317 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1 of peer 1-jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:59,367 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:59,368 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1-jenkins-hbase9.apache.org,44721,1689086199257] zookeeper.ZKUtil(162): syncupReplication1689086213347-0x10154c21b790038, quorum=127.0.0.1:64930, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1689086213354.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0.1689086199666 2023-07-11 14:36:59,370 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086213347-0x10154c21b790038, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689086213354.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,44721,1689086199257/jenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0.1689086199666 2023-07-11 14:36:59,370 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086213347-0x10154c21b790038, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1689086213354.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:59,370 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-0,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase9.apache.org,44721,1689086199257 with the following stats: Total replicated edits: 300, current progress: 2023-07-11 14:36:59,370 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086213347-0x10154c21b790038, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689086213354.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:59,470 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase9.apache.org,44721,1689086199257 2023-07-11 14:36:59,471 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] zookeeper.ZKUtil(243): syncupReplication1689086213347-0x10154c21b790038, quorum=127.0.0.1:64930, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689086213354.syncuptool.replication.org,1234,1/1-jenkins-hbase9.apache.org,44721,1689086199257 because node does not exist (not an error) 2023-07-11 14:36:59,471 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257.replicationSource.shipperjenkins-hbase9.apache.org%2C44721%2C1689086199257.jenkins-hbase9.apache.org%2C44721%2C1689086199257.regiongroup-1,1-jenkins-hbase9.apache.org,44721,1689086199257] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase9.apache.org,44721,1689086199257 with the following stats: Total replicated edits: 300, current progress: 2023-07-11 14:37:02,264 INFO [jenkins-hbase9:40099Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-11 14:37:02,264 INFO [jenkins-hbase9:40099Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-07-11 14:37:03,379 INFO [Listener at localhost/42861] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-11 14:37:03,483 DEBUG [Listener at localhost/42861.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b79003a, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:37:03,483 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2c2ae33d to 127.0.0.1:64930 2023-07-11 14:37:03,484 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:37:03,686 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086213347-0x10154c21b790038, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:37:03,686 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689086213347-0x10154c21b790038, quorum=127.0.0.1:64930, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:37:03,692 INFO [Listener at localhost/42861] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-07-11 14:37:03,693 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-11 14:37:03,693 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x69c030b6 to 127.0.0.1:64930 2023-07-11 14:37:03,693 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:37:03,693 DEBUG [Listener at localhost/42861] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-11 14:37:03,693 DEBUG [Listener at localhost/42861] util.JVMClusterUtil(257): Found active master hash=533785339, stopped=false 2023-07-11 14:37:03,693 INFO [Listener at localhost/42861] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:37:03,695 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-11 14:37:03,695 INFO [Listener at localhost/42861] procedure2.ProcedureExecutor(629): Stopping 2023-07-11 14:37:03,695 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:37:03,695 DEBUG [Listener at localhost/42861] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2490333e to 127.0.0.1:64930 2023-07-11 14:37:03,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:37:03,695 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-11 14:37:03,695 DEBUG [Listener at localhost/42861] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:37:03,699 INFO [Listener at localhost/42861] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,40099,1689086206978' ***** 2023-07-11 14:37:03,699 INFO [Listener at localhost/42861] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-11 14:37:03,700 INFO [RS:0;jenkins-hbase9:40099] regionserver.HeapMemoryManager(220): Stopping 2023-07-11 14:37:03,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-11 14:37:03,700 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-11 14:37:03,700 INFO [RS:0;jenkins-hbase9:40099] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-11 14:37:03,701 INFO [RS:0;jenkins-hbase9:40099] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-11 14:37:03,701 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(3305): Received CLOSE for f27301259d11533bab9412d981c5782d 2023-07-11 14:37:03,701 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(3305): Received CLOSE for d4185bdf44467f073faaa5de49a197d7 2023-07-11 14:37:03,702 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(3305): Received CLOSE for 34f512aa6fa4c9880527ac501e69b8b2 2023-07-11 14:37:03,702 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:37:03,702 DEBUG [RS:0;jenkins-hbase9:40099] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3a739fd8 to 127.0.0.1:64930 2023-07-11 14:37:03,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing f27301259d11533bab9412d981c5782d, disabling compactions & flushes 2023-07-11 14:37:03,702 DEBUG [RS:0;jenkins-hbase9:40099] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:37:03,702 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:37:03,702 INFO [RS:0;jenkins-hbase9:40099] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-11 14:37:03,702 INFO [RS:0;jenkins-hbase9:40099] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-11 14:37:03,702 INFO [RS:0;jenkins-hbase9:40099] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-11 14:37:03,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:37:03,703 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-11 14:37:03,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. after waiting 0 ms 2023-07-11 14:37:03,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:37:03,703 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-11 14:37:03,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing f27301259d11533bab9412d981c5782d 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-11 14:37:03,703 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-11 14:37:03,704 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-11 14:37:03,704 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-11 14:37:03,704 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-11 14:37:03,704 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-11 14:37:03,704 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-11 14:37:03,703 DEBUG [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1478): Online Regions={f27301259d11533bab9412d981c5782d=t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d., 1588230740=hbase:meta,,1.1588230740, d4185bdf44467f073faaa5de49a197d7=hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7., 34f512aa6fa4c9880527ac501e69b8b2=t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2.} 2023-07-11 14:37:03,706 DEBUG [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1504): Waiting on 1588230740, 34f512aa6fa4c9880527ac501e69b8b2, d4185bdf44467f073faaa5de49a197d7, f27301259d11533bab9412d981c5782d 2023-07-11 14:37:03,729 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/.tmp/info/46e3905d6d6343b7acf6d3c5f1a02e3d 2023-07-11 14:37:03,738 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/.tmp/cf1/82aafa819b8f48ce8808b8f9d18e62b6 2023-07-11 14:37:03,744 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/.tmp/info/46e3905d6d6343b7acf6d3c5f1a02e3d as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info/46e3905d6d6343b7acf6d3c5f1a02e3d 2023-07-11 14:37:03,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/.tmp/cf1/82aafa819b8f48ce8808b8f9d18e62b6 as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1/82aafa819b8f48ce8808b8f9d18e62b6 2023-07-11 14:37:03,749 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/info/46e3905d6d6343b7acf6d3c5f1a02e3d, entries=24, sequenceid=43, filesize=7.4 K 2023-07-11 14:37:03,750 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/cf1/82aafa819b8f48ce8808b8f9d18e62b6, entries=100, sequenceid=21, filesize=8.3 K 2023-07-11 14:37:03,750 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 46ms, sequenceid=43, compaction requested=true 2023-07-11 14:37:03,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for f27301259d11533bab9412d981c5782d in 51ms, sequenceid=21, compaction requested=true 2023-07-11 14:37:03,756 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:37:03,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t1_syncup/f27301259d11533bab9412d981c5782d/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-07-11 14:37:03,773 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-07-11 14:37:03,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:37:03,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for f27301259d11533bab9412d981c5782d: 2023-07-11 14:37:03,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689086166686.f27301259d11533bab9412d981c5782d. 2023-07-11 14:37:03,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing d4185bdf44467f073faaa5de49a197d7, disabling compactions & flushes 2023-07-11 14:37:03,774 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:37:03,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:37:03,774 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-11 14:37:03,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. after waiting 0 ms 2023-07-11 14:37:03,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:37:03,775 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-11 14:37:03,775 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-11 14:37:03,775 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-11 14:37:03,781 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/hbase/namespace/d4185bdf44467f073faaa5de49a197d7/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-07-11 14:37:03,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:37:03,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for d4185bdf44467f073faaa5de49a197d7: 2023-07-11 14:37:03,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689086164763.d4185bdf44467f073faaa5de49a197d7. 2023-07-11 14:37:03,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 34f512aa6fa4c9880527ac501e69b8b2, disabling compactions & flushes 2023-07-11 14:37:03,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:37:03,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:37:03,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. after waiting 0 ms 2023-07-11 14:37:03,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:37:03,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 34f512aa6fa4c9880527ac501e69b8b2 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-11 14:37:03,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/.tmp/cf1/826b93ffd19b41e3894dafb4fc4b66bb 2023-07-11 14:37:03,806 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/.tmp/cf1/826b93ffd19b41e3894dafb4fc4b66bb as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1/826b93ffd19b41e3894dafb4fc4b66bb 2023-07-11 14:37:03,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/cf1/826b93ffd19b41e3894dafb4fc4b66bb, entries=200, sequenceid=22, filesize=12.2 K 2023-07-11 14:37:03,811 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 34f512aa6fa4c9880527ac501e69b8b2 in 28ms, sequenceid=22, compaction requested=true 2023-07-11 14:37:03,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/data/default/t2_syncup/34f512aa6fa4c9880527ac501e69b8b2/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=18 2023-07-11 14:37:03,827 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:37:03,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 34f512aa6fa4c9880527ac501e69b8b2: 2023-07-11 14:37:03,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689086167308.34f512aa6fa4c9880527ac501e69b8b2. 2023-07-11 14:37:03,906 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,40099,1689086206978; all regions closed. 2023-07-11 14:37:03,906 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:37:03,914 DEBUG [RS:0;jenkins-hbase9:40099] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:37:03,914 INFO [RS:0;jenkins-hbase9:40099] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C40099%2C1689086206978.meta:.meta(num 1689086207500) 2023-07-11 14:37:03,914 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:37:03,921 DEBUG [RS:0;jenkins-hbase9:40099] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:37:03,921 INFO [RS:0;jenkins-hbase9:40099] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C40099%2C1689086206978.jenkins-hbase9.apache.org%2C40099%2C1689086206978.regiongroup-1:(num 1689086212039) 2023-07-11 14:37:03,921 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/WALs/jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:37:03,928 DEBUG [RS:0;jenkins-hbase9:40099] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/oldWALs 2023-07-11 14:37:03,928 INFO [RS:0;jenkins-hbase9:40099] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase9.apache.org%2C40099%2C1689086206978.jenkins-hbase9.apache.org%2C40099%2C1689086206978.regiongroup-0:(num 1689086207369) 2023-07-11 14:37:03,928 DEBUG [RS:0;jenkins-hbase9:40099] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:37:03,928 INFO [RS:0;jenkins-hbase9:40099] regionserver.LeaseManager(133): Closed leases 2023-07-11 14:37:03,928 INFO [RS:0;jenkins-hbase9:40099] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-11 14:37:03,928 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:37:03,928 DEBUG [RS:0;jenkins-hbase9:40099] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4ac85646 to 127.0.0.1:64930 2023-07-11 14:37:03,928 DEBUG [RS:0;jenkins-hbase9:40099] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:37:03,929 INFO [RS:0;jenkins-hbase9:40099] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:40099 2023-07-11 14:37:03,932 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:37:03,932 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase9.apache.org,40099,1689086206978 2023-07-11 14:37:03,932 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b79003c, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:37:03,932 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:37:03,932 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10154c21b790026, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-11 14:37:03,932 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-11 14:37:03,932 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-11 14:37:03,932 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,40099,1689086206978] 2023-07-11 14:37:03,932 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,40099,1689086206978; numProcessing=1 2023-07-11 14:37:03,935 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase9.apache.org,40099,1689086206978 already deleted, retry=false 2023-07-11 14:37:03,935 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,40099,1689086206978 expired; onlineServers=0 2023-07-11 14:37:03,935 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase9.apache.org,38725,1689086206943' ***** 2023-07-11 14:37:03,935 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-11 14:37:03,935 DEBUG [M:0;jenkins-hbase9:38725] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1bef6dd9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-07-11 14:37:03,935 INFO [M:0;jenkins-hbase9:38725] regionserver.HRegionServer(1144): stopping server jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:37:03,935 INFO [M:0;jenkins-hbase9:38725] regionserver.HRegionServer(1170): stopping server jenkins-hbase9.apache.org,38725,1689086206943; all regions closed. 2023-07-11 14:37:03,935 DEBUG [M:0;jenkins-hbase9:38725] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-11 14:37:03,935 DEBUG [M:0;jenkins-hbase9:38725] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-11 14:37:03,936 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-11 14:37:03,936 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086207170] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1689086207170,5,FailOnTimeoutGroup] 2023-07-11 14:37:03,936 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086207173] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1689086207173,5,FailOnTimeoutGroup] 2023-07-11 14:37:03,936 DEBUG [M:0;jenkins-hbase9:38725] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-11 14:37:03,937 INFO [M:0;jenkins-hbase9:38725] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-11 14:37:03,937 INFO [M:0;jenkins-hbase9:38725] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-11 14:37:03,937 INFO [M:0;jenkins-hbase9:38725] hbase.ChoreService(369): Chore service for: master/jenkins-hbase9:0 had [] on shutdown 2023-07-11 14:37:03,937 DEBUG [M:0;jenkins-hbase9:38725] master.HMaster(1512): Stopping service threads 2023-07-11 14:37:03,937 INFO [M:0;jenkins-hbase9:38725] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-11 14:37:03,938 ERROR [M:0;jenkins-hbase9:38725] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-11 14:37:03,938 INFO [M:0;jenkins-hbase9:38725] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-11 14:37:03,938 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-11 14:37:03,938 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-11 14:37:03,938 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-11 14:37:03,939 DEBUG [M:0;jenkins-hbase9:38725] zookeeper.ZKUtil(398): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-11 14:37:03,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-11 14:37:03,939 WARN [M:0;jenkins-hbase9:38725] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-11 14:37:03,939 INFO [M:0;jenkins-hbase9:38725] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-11 14:37:03,939 INFO [M:0;jenkins-hbase9:38725] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-11 14:37:03,940 DEBUG [M:0;jenkins-hbase9:38725] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-11 14:37:03,940 INFO [M:0;jenkins-hbase9:38725] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:37:03,940 DEBUG [M:0;jenkins-hbase9:38725] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:37:03,940 DEBUG [M:0;jenkins-hbase9:38725] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-11 14:37:03,940 DEBUG [M:0;jenkins-hbase9:38725] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:37:03,940 INFO [M:0;jenkins-hbase9:38725] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-07-11 14:37:03,953 INFO [M:0;jenkins-hbase9:38725] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=309 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/46aaef71bd904abd8d0bc417c5869354 2023-07-11 14:37:03,958 DEBUG [M:0;jenkins-hbase9:38725] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/46aaef71bd904abd8d0bc417c5869354 as hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/46aaef71bd904abd8d0bc417c5869354 2023-07-11 14:37:03,963 INFO [M:0;jenkins-hbase9:38725] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/46aaef71bd904abd8d0bc417c5869354, entries=9, sequenceid=309, filesize=5.5 K 2023-07-11 14:37:03,964 INFO [M:0;jenkins-hbase9:38725] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=309, compaction requested=true 2023-07-11 14:37:03,964 INFO [M:0;jenkins-hbase9:38725] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-11 14:37:03,964 DEBUG [M:0;jenkins-hbase9:38725] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-11 14:37:03,965 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/b8488d2f-9f76-a0e2-98ab-5dee710e0902/MasterData/WALs/jenkins-hbase9.apache.org,38725,1689086206943 2023-07-11 14:37:03,972 INFO [M:0;jenkins-hbase9:38725] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-11 14:37:03,972 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-11 14:37:03,973 INFO [M:0;jenkins-hbase9:38725] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:38725 2023-07-11 14:37:03,975 DEBUG [M:0;jenkins-hbase9:38725] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase9.apache.org,38725,1689086206943 already deleted, retry=false 2023-07-11 14:37:04,095 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:37:04,095 INFO [M:0;jenkins-hbase9:38725] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,38725,1689086206943; zookeeper connection closed. 2023-07-11 14:37:04,095 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): master:38725-0x10154c21b790032, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:37:04,195 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:37:04,195 INFO [RS:0;jenkins-hbase9:40099] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase9.apache.org,40099,1689086206978; zookeeper connection closed. 2023-07-11 14:37:04,195 DEBUG [Listener at localhost/42861-EventThread] zookeeper.ZKWatcher(600): regionserver:40099-0x10154c21b790033, quorum=127.0.0.1:64930, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-11 14:37:04,196 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5816015f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5816015f 2023-07-11 14:37:04,196 INFO [Listener at localhost/42861] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-11 14:37:04,196 WARN [Listener at localhost/42861] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-11 14:37:04,207 INFO [Listener at localhost/42861] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-11 14:37:04,312 WARN [BP-1145119001-172.31.2.10-1689086162520 heartbeating to localhost/127.0.0.1:33985] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-11 14:37:04,312 WARN [BP-1145119001-172.31.2.10-1689086162520 heartbeating to localhost/127.0.0.1:33985] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1145119001-172.31.2.10-1689086162520 (Datanode Uuid 3a595fa8-21e9-4073-8979-ed9ec626fc3a) service to localhost/127.0.0.1:33985 2023-07-11 14:37:04,314 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/cluster_dc28c8bb-79eb-6d50-ac76-892a20519f73/dfs/data/data7/current/BP-1145119001-172.31.2.10-1689086162520] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:04,314 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/cluster_dc28c8bb-79eb-6d50-ac76-892a20519f73/dfs/data/data8/current/BP-1145119001-172.31.2.10-1689086162520] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:04,316 WARN [Listener at localhost/42861] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-11 14:37:04,321 INFO [Listener at localhost/42861] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-11 14:37:04,427 WARN [BP-1145119001-172.31.2.10-1689086162520 heartbeating to localhost/127.0.0.1:33985] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-11 14:37:04,427 WARN [BP-1145119001-172.31.2.10-1689086162520 heartbeating to localhost/127.0.0.1:33985] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1145119001-172.31.2.10-1689086162520 (Datanode Uuid a7aa9ac2-c7cc-4611-a08f-9ba7ee8081a7) service to localhost/127.0.0.1:33985 2023-07-11 14:37:04,428 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/cluster_dc28c8bb-79eb-6d50-ac76-892a20519f73/dfs/data/data5/current/BP-1145119001-172.31.2.10-1689086162520] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:04,429 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/cluster_dc28c8bb-79eb-6d50-ac76-892a20519f73/dfs/data/data6/current/BP-1145119001-172.31.2.10-1689086162520] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:04,431 WARN [Listener at localhost/42861] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-11 14:37:04,436 INFO [Listener at localhost/42861] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-11 14:37:04,540 WARN [BP-1145119001-172.31.2.10-1689086162520 heartbeating to localhost/127.0.0.1:33985] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-11 14:37:04,540 WARN [BP-1145119001-172.31.2.10-1689086162520 heartbeating to localhost/127.0.0.1:33985] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1145119001-172.31.2.10-1689086162520 (Datanode Uuid 5295bcd3-d2e9-45a1-ade9-3508dd85f33a) service to localhost/127.0.0.1:33985 2023-07-11 14:37:04,541 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/cluster_dc28c8bb-79eb-6d50-ac76-892a20519f73/dfs/data/data3/current/BP-1145119001-172.31.2.10-1689086162520] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:04,541 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/cluster_dc28c8bb-79eb-6d50-ac76-892a20519f73/dfs/data/data4/current/BP-1145119001-172.31.2.10-1689086162520] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:04,542 WARN [Listener at localhost/42861] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-11 14:37:04,545 INFO [Listener at localhost/42861] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-11 14:37:04,649 WARN [BP-1145119001-172.31.2.10-1689086162520 heartbeating to localhost/127.0.0.1:33985] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-11 14:37:04,649 WARN [BP-1145119001-172.31.2.10-1689086162520 heartbeating to localhost/127.0.0.1:33985] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1145119001-172.31.2.10-1689086162520 (Datanode Uuid 2c80c93b-7350-4238-b8c3-7d9d0ea5544f) service to localhost/127.0.0.1:33985 2023-07-11 14:37:04,650 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/cluster_dc28c8bb-79eb-6d50-ac76-892a20519f73/dfs/data/data1/current/BP-1145119001-172.31.2.10-1689086162520] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:04,650 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b2ee369-c86f-4992-9c5a-b249523cb12e/cluster_dc28c8bb-79eb-6d50-ac76-892a20519f73/dfs/data/data2/current/BP-1145119001-172.31.2.10-1689086162520] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:04,677 INFO [Listener at localhost/42861] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-11 14:37:04,864 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-11 14:37:04,864 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-11 14:37:04,865 WARN [Listener at localhost/42861] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-11 14:37:04,867 INFO [Listener at localhost/42861] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-11 14:37:04,869 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-11 14:37:04,969 WARN [BP-211014841-172.31.2.10-1689086155813 heartbeating to localhost/127.0.0.1:41201] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-11 14:37:04,969 WARN [BP-211014841-172.31.2.10-1689086155813 heartbeating to localhost/127.0.0.1:41201] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-211014841-172.31.2.10-1689086155813 (Datanode Uuid 5d203a92-b49d-4b88-b811-a29d937209c2) service to localhost/127.0.0.1:41201 2023-07-11 14:37:04,970 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/cluster_5dfd46ef-8369-c534-0644-37f0dfe3e9ac/dfs/data/data3/current/BP-211014841-172.31.2.10-1689086155813] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:04,970 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/cluster_5dfd46ef-8369-c534-0644-37f0dfe3e9ac/dfs/data/data4/current/BP-211014841-172.31.2.10-1689086155813] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:04,971 WARN [Listener at localhost/42861] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-11 14:37:04,973 INFO [Listener at localhost/42861] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-11 14:37:05,075 WARN [BP-211014841-172.31.2.10-1689086155813 heartbeating to localhost/127.0.0.1:41201] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-11 14:37:05,075 WARN [BP-211014841-172.31.2.10-1689086155813 heartbeating to localhost/127.0.0.1:41201] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-211014841-172.31.2.10-1689086155813 (Datanode Uuid af9149e4-ca9e-41bc-b3e9-47b65276d12a) service to localhost/127.0.0.1:41201 2023-07-11 14:37:05,076 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/cluster_5dfd46ef-8369-c534-0644-37f0dfe3e9ac/dfs/data/data1/current/BP-211014841-172.31.2.10-1689086155813] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:05,076 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f45a2fe9-9e55-da39-874a-76989d1df794/cluster_5dfd46ef-8369-c534-0644-37f0dfe3e9ac/dfs/data/data2/current/BP-211014841-172.31.2.10-1689086155813] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-11 14:37:05,085 INFO [Listener at localhost/42861] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-11 14:37:05,203 INFO [Listener at localhost/42861] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-11 14:37:05,227 INFO [Listener at localhost/42861] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-11 14:37:05,248 INFO [Listener at localhost/42861] hbase.ResourceChecker(175): after: replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL#testSyncUpTool Thread=183 (was 10) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:33985 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x1dfecc23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/2072866908.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x09d430cc-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: IPC Client (1837182992) connection to localhost/127.0.0.1:33985 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:41201 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x09d430cc sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/2072866908.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase9:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x709fcb85-SendThread(127.0.0.1:64930) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1837182992) connection to localhost/127.0.0.1:33985 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x709fcb85-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase9:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Client (1837182992) connection to localhost/127.0.0.1:41201 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:33985 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase9:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:33985 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: master/jenkins-hbase9:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase9:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-3-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x14446692-SendThread(127.0.0.1:64930) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:41201 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x14446692 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/2072866908.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase9:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/42861 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:33985 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase9:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x1dfecc23-SendThread(127.0.0.1:64930) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x2a94bf46-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x09d430cc-SendThread(127.0.0.1:64930) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1837182992) connection to localhost/127.0.0.1:41201 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@61dfd54f java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x2a94bf46-SendThread(127.0.0.1:64930) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:41201 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x709fcb85 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/2072866908.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x2a94bf46 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/2072866908.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase9:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,42359,1689086159924-SendThread(127.0.0.1:64930) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:33985 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:41201 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x1dfecc23-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase9:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:33985 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:33985 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1837182992) connection to localhost/127.0.0.1:33985 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64930@0x14446692-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: regionserver/jenkins-hbase9:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,37647,1689086159883-SendThread(127.0.0.1:64930) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase9.apache.org,44721,1689086199257-SendThread(127.0.0.1:64930) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: regionserver/jenkins-hbase9:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) - Thread LEAK? -, OpenFileDescriptor=702 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=180 (was 229), ProcessCount=172 (was 172), AvailableMemoryMB=6730 (was 7818)