2023-08-17 16:56:12,807 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1 2023-08-17 16:56:12,815 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-log-dir so I do NOT create it in target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8 2023-08-17 16:56:12,816 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-log-dir Erasing configuration value by system value. 2023-08-17 16:56:12,816 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-tmp-dir so I do NOT create it in target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8 2023-08-17 16:56:12,816 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-tmp-dir Erasing configuration value by system value. 2023-08-17 16:56:12,816 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8 2023-08-17 16:56:12,829 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-08-17 16:56:12,830 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL timeout: 13 mins 2023-08-17 16:56:12,860 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=175, ProcessCount=170, AvailableMemoryMB=5184 2023-08-17 16:56:12,871 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-log-dir so I do NOT create it in target/test-data/7e4b2a21-0663-b1a8-793b-f0377e34d7e5 2023-08-17 16:56:12,871 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-log-dir Erasing configuration value by system value. 2023-08-17 16:56:12,871 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-tmp-dir so I do NOT create it in target/test-data/7e4b2a21-0663-b1a8-793b-f0377e34d7e5 2023-08-17 16:56:12,871 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-tmp-dir Erasing configuration value by system value. 2023-08-17 16:56:12,872 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7e4b2a21-0663-b1a8-793b-f0377e34d7e5 2023-08-17 16:56:12,881 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-log-dir so I do NOT create it in target/test-data/c457571c-4fcd-2691-f499-b997458a5b6d 2023-08-17 16:56:12,882 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-log-dir Erasing configuration value by system value. 2023-08-17 16:56:12,882 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-tmp-dir so I do NOT create it in target/test-data/c457571c-4fcd-2691-f499-b997458a5b6d 2023-08-17 16:56:12,882 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop-tmp-dir Erasing configuration value by system value. 2023-08-17 16:56:12,883 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c457571c-4fcd-2691-f499-b997458a5b6d 2023-08-17 16:56:12,887 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/cluster_72c0133a-0ba1-29b3-3528-a3510554313d, deleteOnExit=true 2023-08-17 16:56:13,022 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/cluster_72c0133a-0ba1-29b3-3528-a3510554313d/zookeeper_0, clientPort=56119, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/cluster_72c0133a-0ba1-29b3-3528-a3510554313d/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/cluster_72c0133a-0ba1-29b3-3528-a3510554313d/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-17 16:56:13,039 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=56119 2023-08-17 16:56:13,044 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-17 16:56:13,044 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-17 16:56:13,044 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/test.cache.data in system properties and HBase conf 2023-08-17 16:56:13,045 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop.tmp.dir in system properties and HBase conf 2023-08-17 16:56:13,045 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/hadoop.log.dir in system properties and HBase conf 2023-08-17 16:56:13,045 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-17 16:56:13,046 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-17 16:56:13,046 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-17 16:56:13,143 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-08-17 16:56:13,488 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-17 16:56:13,492 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-17 16:56:13,492 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-17 16:56:13,493 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-17 16:56:13,493 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-17 16:56:13,493 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-17 16:56:13,493 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-17 16:56:13,494 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-17 16:56:13,494 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-17 16:56:13,494 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-17 16:56:13,494 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/nfs.dump.dir in system properties and HBase conf 2023-08-17 16:56:13,495 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/java.io.tmpdir in system properties and HBase conf 2023-08-17 16:56:13,495 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-17 16:56:13,495 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-17 16:56:13,495 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-17 16:56:13,973 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-17 16:56:13,978 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-17 16:56:14,248 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-08-17 16:56:14,364 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-08-17 16:56:14,379 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-17 16:56:14,419 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-08-17 16:56:14,452 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/java.io.tmpdir/Jetty_localhost_36829_hdfs____.ovfswq/webapp 2023-08-17 16:56:14,596 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36829 2023-08-17 16:56:14,614 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-17 16:56:14,615 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-17 16:56:15,082 WARN [Listener at localhost/38511] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-17 16:56:15,144 WARN [Listener at localhost/38511] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-17 16:56:15,161 WARN [Listener at localhost/38511] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-17 16:56:15,167 INFO [Listener at localhost/38511] log.Slf4jLog(67): jetty-6.1.26 2023-08-17 16:56:15,171 INFO [Listener at localhost/38511] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/java.io.tmpdir/Jetty_localhost_34145_datanode____.4r9ulk/webapp 2023-08-17 16:56:15,266 INFO [Listener at localhost/38511] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34145 2023-08-17 16:56:15,567 WARN [Listener at localhost/43631] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-17 16:56:15,580 WARN [Listener at localhost/43631] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-17 16:56:15,583 WARN [Listener at localhost/43631] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-17 16:56:15,585 INFO [Listener at localhost/43631] log.Slf4jLog(67): jetty-6.1.26 2023-08-17 16:56:15,594 INFO [Listener at localhost/43631] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/java.io.tmpdir/Jetty_localhost_33995_datanode____j9xpuk/webapp 2023-08-17 16:56:15,693 INFO [Listener at localhost/43631] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33995 2023-08-17 16:56:15,703 WARN [Listener at localhost/44847] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-17 16:56:16,012 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb3ca24a58274124b: Processing first storage report for DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d from datanode 173bd245-09c0-4c76-92f9-36a923215fca 2023-08-17 16:56:16,014 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb3ca24a58274124b: from storage DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d node DatanodeRegistration(127.0.0.1:39023, datanodeUuid=173bd245-09c0-4c76-92f9-36a923215fca, infoPort=36763, infoSecurePort=0, ipcPort=44847, storageInfo=lv=-57;cid=testClusterID;nsid=1239877717;c=1692291374046), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-17 16:56:16,014 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x1f3822a4257ec5c: Processing first storage report for DS-014fa655-a3a0-4df1-a42d-845ba3b78c73 from datanode e47db782-26c3-4869-b653-86aa503a6c38 2023-08-17 16:56:16,014 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x1f3822a4257ec5c: from storage DS-014fa655-a3a0-4df1-a42d-845ba3b78c73 node DatanodeRegistration(127.0.0.1:43377, datanodeUuid=e47db782-26c3-4869-b653-86aa503a6c38, infoPort=42385, infoSecurePort=0, ipcPort=43631, storageInfo=lv=-57;cid=testClusterID;nsid=1239877717;c=1692291374046), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-17 16:56:16,014 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb3ca24a58274124b: Processing first storage report for DS-f2f79e22-5a7c-4adc-b514-027d2d01b07c from datanode 173bd245-09c0-4c76-92f9-36a923215fca 2023-08-17 16:56:16,014 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb3ca24a58274124b: from storage DS-f2f79e22-5a7c-4adc-b514-027d2d01b07c node DatanodeRegistration(127.0.0.1:39023, datanodeUuid=173bd245-09c0-4c76-92f9-36a923215fca, infoPort=36763, infoSecurePort=0, ipcPort=44847, storageInfo=lv=-57;cid=testClusterID;nsid=1239877717;c=1692291374046), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-17 16:56:16,015 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x1f3822a4257ec5c: Processing first storage report for DS-38c6afed-60bc-4ef2-9390-079d8e87754c from datanode e47db782-26c3-4869-b653-86aa503a6c38 2023-08-17 16:56:16,015 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x1f3822a4257ec5c: from storage DS-38c6afed-60bc-4ef2-9390-079d8e87754c node DatanodeRegistration(127.0.0.1:43377, datanodeUuid=e47db782-26c3-4869-b653-86aa503a6c38, infoPort=42385, infoSecurePort=0, ipcPort=43631, storageInfo=lv=-57;cid=testClusterID;nsid=1239877717;c=1692291374046), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-17 16:56:16,090 DEBUG [Listener at localhost/44847] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1 2023-08-17 16:56:16,098 INFO [Listener at localhost/44847] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:16,100 INFO [Listener at localhost/44847] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:16,756 INFO [Listener at localhost/44847] util.FSUtils(471): Created version file at hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85 with version=8 2023-08-17 16:56:16,757 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/hbase-staging 2023-08-17 16:56:17,071 INFO [Listener at localhost/44847] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-08-17 16:56:17,535 INFO [Listener at localhost/44847] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-17 16:56:17,565 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:17,566 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:17,566 INFO [Listener at localhost/44847] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:17,566 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:17,566 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:17,702 INFO [Listener at localhost/44847] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:17,772 DEBUG [Listener at localhost/44847] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-08-17 16:56:17,865 INFO [Listener at localhost/44847] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33151 2023-08-17 16:56:17,875 INFO [Listener at localhost/44847] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:17,877 INFO [Listener at localhost/44847] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:17,902 INFO [Listener at localhost/44847] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33151 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:17,945 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:331510x0, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:17,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33151-0x1001b2c4f410000 connected 2023-08-17 16:56:17,970 DEBUG [Listener at localhost/44847] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-17 16:56:17,971 DEBUG [Listener at localhost/44847] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-17 16:56:17,974 DEBUG [Listener at localhost/44847] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-17 16:56:17,982 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33151 2023-08-17 16:56:17,982 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33151 2023-08-17 16:56:17,983 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33151 2023-08-17 16:56:17,983 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33151 2023-08-17 16:56:17,984 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33151 2023-08-17 16:56:17,988 INFO [Listener at localhost/44847] master.HMaster(444): hbase.rootdir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85, hbase.cluster.distributed=false 2023-08-17 16:56:18,054 INFO [Listener at localhost/44847] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-17 16:56:18,054 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:18,054 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:18,054 INFO [Listener at localhost/44847] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:18,055 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:18,055 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:18,059 INFO [Listener at localhost/44847] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:18,062 INFO [Listener at localhost/44847] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45379 2023-08-17 16:56:18,064 INFO [Listener at localhost/44847] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-17 16:56:18,070 DEBUG [Listener at localhost/44847] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-17 16:56:18,071 INFO [Listener at localhost/44847] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:18,073 INFO [Listener at localhost/44847] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:18,074 INFO [Listener at localhost/44847] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45379 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:18,078 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:453790x0, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:18,079 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45379-0x1001b2c4f410001 connected 2023-08-17 16:56:18,079 DEBUG [Listener at localhost/44847] zookeeper.ZKUtil(164): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-17 16:56:18,080 DEBUG [Listener at localhost/44847] zookeeper.ZKUtil(164): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-17 16:56:18,081 DEBUG [Listener at localhost/44847] zookeeper.ZKUtil(164): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-17 16:56:18,081 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45379 2023-08-17 16:56:18,082 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45379 2023-08-17 16:56:18,082 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45379 2023-08-17 16:56:18,082 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45379 2023-08-17 16:56:18,083 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45379 2023-08-17 16:56:18,095 INFO [Listener at localhost/44847] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-17 16:56:18,096 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:18,096 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:18,096 INFO [Listener at localhost/44847] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:18,096 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:18,097 INFO [Listener at localhost/44847] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:18,097 INFO [Listener at localhost/44847] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:18,098 INFO [Listener at localhost/44847] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42469 2023-08-17 16:56:18,098 INFO [Listener at localhost/44847] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-17 16:56:18,099 DEBUG [Listener at localhost/44847] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-17 16:56:18,100 INFO [Listener at localhost/44847] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:18,102 INFO [Listener at localhost/44847] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:18,103 INFO [Listener at localhost/44847] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42469 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:18,108 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:424690x0, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:18,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42469-0x1001b2c4f410002 connected 2023-08-17 16:56:18,109 DEBUG [Listener at localhost/44847] zookeeper.ZKUtil(164): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-17 16:56:18,109 DEBUG [Listener at localhost/44847] zookeeper.ZKUtil(164): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-17 16:56:18,110 DEBUG [Listener at localhost/44847] zookeeper.ZKUtil(164): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-17 16:56:18,110 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42469 2023-08-17 16:56:18,111 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42469 2023-08-17 16:56:18,111 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42469 2023-08-17 16:56:18,111 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42469 2023-08-17 16:56:18,112 DEBUG [Listener at localhost/44847] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42469 2023-08-17 16:56:18,113 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,33151,1692291376904 2023-08-17 16:56:18,122 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-17 16:56:18,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,33151,1692291376904 2023-08-17 16:56:18,142 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-17 16:56:18,142 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-17 16:56:18,142 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-17 16:56:18,142 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:18,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-17 16:56:18,144 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,33151,1692291376904 from backup master directory 2023-08-17 16:56:18,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-17 16:56:18,147 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,33151,1692291376904 2023-08-17 16:56:18,147 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-17 16:56:18,148 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:18,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33151,1692291376904 2023-08-17 16:56:18,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-08-17 16:56:18,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-08-17 16:56:18,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/hbase.id with ID: 1067016f-f973-4161-8f76-366ce33cec43 2023-08-17 16:56:18,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:18,296 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:18,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3dedaa22 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:18,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1cb28ff9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:18,393 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:18,394 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-17 16:56:18,403 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:18,406 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:18,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store-tmp 2023-08-17 16:56:18,469 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:18,469 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-17 16:56:18,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:18,469 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:18,469 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-17 16:56:18,469 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:18,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:18,470 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:56:18,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,33151,1692291376904 2023-08-17 16:56:18,473 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:18,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33151%2C1692291376904.jenkins-hbase4.apache.org%2C33151%2C1692291376904.regiongroup-0, suffix=, logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,33151,1692291376904, archiveDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/oldWALs, maxLogs=10 2023-08-17 16:56:18,525 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-08-17 16:56:18,525 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-08-17 16:56:18,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:195) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:215) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-17 16:56:18,558 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-17 16:56:18,560 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK] 2023-08-17 16:56:18,560 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK] 2023-08-17 16:56:18,567 DEBUG [RS-EventLoopGroup-4-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-17 16:56:18,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,33151,1692291376904/jenkins-hbase4.apache.org%2C33151%2C1692291376904.jenkins-hbase4.apache.org%2C33151%2C1692291376904.regiongroup-0.1692291378504 2023-08-17 16:56:18,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK], DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK]] 2023-08-17 16:56:18,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:18,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:18,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:18,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:18,692 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:18,700 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-17 16:56:18,725 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-17 16:56:18,737 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:18,742 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:18,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:18,760 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:18,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-17 16:56:18,765 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59167941, jitterRate=-0.11832897365093231}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-17 16:56:18,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:56:18,766 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-17 16:56:18,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-17 16:56:18,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-17 16:56:18,787 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-17 16:56:18,788 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-17 16:56:18,819 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-08-17 16:56:18,819 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-17 16:56:18,843 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-17 16:56:18,847 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-17 16:56:18,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-17 16:56:18,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-17 16:56:18,875 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-08-17 16:56:18,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-17 16:56:18,883 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-08-17 16:56:18,886 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:18,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-08-17 16:56:18,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-08-17 16:56:18,899 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-08-17 16:56:18,904 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-17 16:56:18,904 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-17 16:56:18,904 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-17 16:56:18,904 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:18,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33151,1692291376904, sessionid=0x1001b2c4f410000, setting cluster-up flag (Was=false) 2023-08-17 16:56:18,919 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:18,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-08-17 16:56:18,927 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33151,1692291376904 2023-08-17 16:56:18,933 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:18,938 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-08-17 16:56:18,939 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33151,1692291376904 2023-08-17 16:56:18,941 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.hbase-snapshot/.tmp 2023-08-17 16:56:19,015 INFO [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(951): ClusterId : 1067016f-f973-4161-8f76-366ce33cec43 2023-08-17 16:56:19,015 INFO [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(951): ClusterId : 1067016f-f973-4161-8f76-366ce33cec43 2023-08-17 16:56:19,021 DEBUG [RS:0;jenkins-hbase4:45379] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-17 16:56:19,021 DEBUG [RS:1;jenkins-hbase4:42469] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-17 16:56:19,028 DEBUG [RS:0;jenkins-hbase4:45379] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-17 16:56:19,028 DEBUG [RS:1;jenkins-hbase4:42469] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-17 16:56:19,028 DEBUG [RS:0;jenkins-hbase4:45379] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-17 16:56:19,028 DEBUG [RS:1;jenkins-hbase4:42469] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-17 16:56:19,032 DEBUG [RS:0;jenkins-hbase4:45379] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-17 16:56:19,032 DEBUG [RS:1;jenkins-hbase4:42469] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-17 16:56:19,033 DEBUG [RS:0;jenkins-hbase4:45379] zookeeper.ReadOnlyZKClient(139): Connect 0x794783db to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:19,034 DEBUG [RS:1;jenkins-hbase4:42469] zookeeper.ReadOnlyZKClient(139): Connect 0x76311fa0 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:19,044 DEBUG [RS:0;jenkins-hbase4:45379] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77807283, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:19,044 DEBUG [RS:1;jenkins-hbase4:42469] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78f6ff16, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:19,044 DEBUG [RS:0;jenkins-hbase4:45379] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73bfe8ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:56:19,045 DEBUG [RS:1;jenkins-hbase4:42469] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7cc974d0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:56:19,056 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-17 16:56:19,066 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:19,066 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:19,066 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:19,066 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:19,066 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-17 16:56:19,066 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,067 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:19,067 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,067 DEBUG [RS:1;jenkins-hbase4:42469] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:42469 2023-08-17 16:56:19,067 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:45379 2023-08-17 16:56:19,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692291409069 2023-08-17 16:56:19,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-17 16:56:19,072 INFO [RS:1;jenkins-hbase4:42469] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-17 16:56:19,072 INFO [RS:0;jenkins-hbase4:45379] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-17 16:56:19,073 INFO [RS:0;jenkins-hbase4:45379] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-17 16:56:19,073 INFO [RS:1;jenkins-hbase4:42469] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-17 16:56:19,073 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1022): About to register with Master. 2023-08-17 16:56:19,073 DEBUG [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1022): About to register with Master. 2023-08-17 16:56:19,076 DEBUG [PEWorker-2] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-17 16:56:19,076 INFO [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33151,1692291376904 with isa=jenkins-hbase4.apache.org/172.31.14.131:45379, startcode=1692291378053 2023-08-17 16:56:19,076 INFO [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33151,1692291376904 with isa=jenkins-hbase4.apache.org/172.31.14.131:42469, startcode=1692291378095 2023-08-17 16:56:19,076 INFO [PEWorker-2] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-17 16:56:19,081 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-17 16:56:19,081 INFO [PEWorker-2] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:19,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-17 16:56:19,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-17 16:56:19,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-17 16:56:19,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-17 16:56:19,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-17 16:56:19,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-17 16:56:19,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-17 16:56:19,096 DEBUG [RS:1;jenkins-hbase4:42469] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-17 16:56:19,096 DEBUG [RS:0;jenkins-hbase4:45379] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-17 16:56:19,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-17 16:56:19,099 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-17 16:56:19,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291379100,5,FailOnTimeoutGroup] 2023-08-17 16:56:19,101 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291379100,5,FailOnTimeoutGroup] 2023-08-17 16:56:19,101 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,101 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-17 16:56:19,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,119 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-17 16:56:19,120 INFO [PEWorker-2] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-17 16:56:19,120 INFO [PEWorker-2] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85 2023-08-17 16:56:19,145 DEBUG [PEWorker-2] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:19,148 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-17 16:56:19,151 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/info 2023-08-17 16:56:19,153 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-17 16:56:19,154 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:19,154 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-17 16:56:19,157 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:56:19,158 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-17 16:56:19,159 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:19,159 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-17 16:56:19,162 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/table 2023-08-17 16:56:19,162 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54659, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-08-17 16:56:19,162 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50549, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-08-17 16:56:19,163 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-17 16:56:19,164 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:19,165 DEBUG [PEWorker-2] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740 2023-08-17 16:56:19,166 DEBUG [PEWorker-2] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740 2023-08-17 16:56:19,171 DEBUG [PEWorker-2] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-17 16:56:19,173 DEBUG [PEWorker-2] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-17 16:56:19,175 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33151] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:19,176 DEBUG [PEWorker-2] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-17 16:56:19,178 INFO [PEWorker-2] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59408191, jitterRate=-0.11474896967411041}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-17 16:56:19,178 DEBUG [PEWorker-2] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-17 16:56:19,178 DEBUG [PEWorker-2] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-17 16:56:19,178 INFO [PEWorker-2] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-17 16:56:19,178 DEBUG [PEWorker-2] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-17 16:56:19,178 DEBUG [PEWorker-2] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-17 16:56:19,179 DEBUG [PEWorker-2] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-17 16:56:19,179 INFO [PEWorker-2] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-17 16:56:19,180 DEBUG [PEWorker-2] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-17 16:56:19,185 DEBUG [PEWorker-2] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-17 16:56:19,186 INFO [PEWorker-2] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-17 16:56:19,187 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33151] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:19,193 DEBUG [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85 2023-08-17 16:56:19,193 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85 2023-08-17 16:56:19,193 DEBUG [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38511 2023-08-17 16:56:19,193 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38511 2023-08-17 16:56:19,193 DEBUG [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-17 16:56:19,193 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-17 16:56:19,195 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-17 16:56:19,198 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-17 16:56:19,199 DEBUG [RS:0;jenkins-hbase4:45379] zookeeper.ZKUtil(162): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:19,199 DEBUG [RS:1;jenkins-hbase4:42469] zookeeper.ZKUtil(162): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:19,199 WARN [RS:0;jenkins-hbase4:45379] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:19,200 WARN [RS:1;jenkins-hbase4:42469] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:19,200 INFO [RS:0;jenkins-hbase4:45379] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:19,200 INFO [RS:1;jenkins-hbase4:42469] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:19,200 INFO [RS:0;jenkins-hbase4:45379] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:19,201 INFO [RS:1;jenkins-hbase4:42469] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:19,201 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1948): logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:19,201 DEBUG [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1948): logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:19,202 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42469,1692291378095] 2023-08-17 16:56:19,202 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45379,1692291378053] 2023-08-17 16:56:19,211 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-17 16:56:19,213 DEBUG [RS:1;jenkins-hbase4:42469] zookeeper.ZKUtil(162): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:19,213 DEBUG [RS:0;jenkins-hbase4:45379] zookeeper.ZKUtil(162): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:19,213 DEBUG [RS:1;jenkins-hbase4:42469] zookeeper.ZKUtil(162): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:19,213 DEBUG [RS:0;jenkins-hbase4:45379] zookeeper.ZKUtil(162): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:19,216 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-17 16:56:19,224 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:56:19,224 DEBUG [RS:1;jenkins-hbase4:42469] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:56:19,232 INFO [RS:1;jenkins-hbase4:42469] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-17 16:56:19,232 INFO [RS:0;jenkins-hbase4:45379] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-17 16:56:19,251 INFO [RS:0;jenkins-hbase4:45379] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-17 16:56:19,251 INFO [RS:1;jenkins-hbase4:42469] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-17 16:56:19,255 INFO [RS:0;jenkins-hbase4:45379] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-17 16:56:19,255 INFO [RS:1;jenkins-hbase4:42469] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-17 16:56:19,255 INFO [RS:0;jenkins-hbase4:45379] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,255 INFO [RS:1;jenkins-hbase4:42469] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,256 INFO [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-17 16:56:19,256 INFO [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-17 16:56:19,263 INFO [RS:1;jenkins-hbase4:42469] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,263 INFO [RS:0;jenkins-hbase4:45379] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,264 DEBUG [RS:1;jenkins-hbase4:42469] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,264 DEBUG [RS:0;jenkins-hbase4:45379] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,264 DEBUG [RS:1;jenkins-hbase4:42469] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,264 DEBUG [RS:0;jenkins-hbase4:45379] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,264 DEBUG [RS:1;jenkins-hbase4:42469] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,264 DEBUG [RS:0;jenkins-hbase4:45379] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,264 DEBUG [RS:1;jenkins-hbase4:42469] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,264 DEBUG [RS:0;jenkins-hbase4:45379] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,264 DEBUG [RS:1;jenkins-hbase4:42469] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,265 DEBUG [RS:0;jenkins-hbase4:45379] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,265 DEBUG [RS:1;jenkins-hbase4:42469] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:19,265 DEBUG [RS:0;jenkins-hbase4:45379] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:19,265 DEBUG [RS:1;jenkins-hbase4:42469] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,265 DEBUG [RS:0;jenkins-hbase4:45379] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,265 DEBUG [RS:1;jenkins-hbase4:42469] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,265 DEBUG [RS:0;jenkins-hbase4:45379] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,265 DEBUG [RS:1;jenkins-hbase4:42469] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,265 DEBUG [RS:0;jenkins-hbase4:45379] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,265 DEBUG [RS:1;jenkins-hbase4:42469] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,265 DEBUG [RS:0;jenkins-hbase4:45379] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:19,266 INFO [RS:0;jenkins-hbase4:45379] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,267 INFO [RS:0;jenkins-hbase4:45379] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,267 INFO [RS:0;jenkins-hbase4:45379] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,267 INFO [RS:1;jenkins-hbase4:42469] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,267 INFO [RS:1;jenkins-hbase4:42469] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,267 INFO [RS:1;jenkins-hbase4:42469] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,282 INFO [RS:0;jenkins-hbase4:45379] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-17 16:56:19,282 INFO [RS:1;jenkins-hbase4:42469] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-17 16:56:19,284 INFO [RS:1;jenkins-hbase4:42469] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42469,1692291378095-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,284 INFO [RS:0;jenkins-hbase4:45379] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45379,1692291378053-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,303 INFO [RS:0;jenkins-hbase4:45379] regionserver.Replication(203): jenkins-hbase4.apache.org,45379,1692291378053 started 2023-08-17 16:56:19,304 INFO [RS:1;jenkins-hbase4:42469] regionserver.Replication(203): jenkins-hbase4.apache.org,42469,1692291378095 started 2023-08-17 16:56:19,304 INFO [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45379,1692291378053, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45379, sessionid=0x1001b2c4f410001 2023-08-17 16:56:19,304 INFO [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42469,1692291378095, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42469, sessionid=0x1001b2c4f410002 2023-08-17 16:56:19,304 DEBUG [RS:0;jenkins-hbase4:45379] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-17 16:56:19,304 DEBUG [RS:1;jenkins-hbase4:42469] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-17 16:56:19,304 DEBUG [RS:0;jenkins-hbase4:45379] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:19,304 DEBUG [RS:1;jenkins-hbase4:42469] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:19,305 DEBUG [RS:0;jenkins-hbase4:45379] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45379,1692291378053' 2023-08-17 16:56:19,305 DEBUG [RS:1;jenkins-hbase4:42469] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42469,1692291378095' 2023-08-17 16:56:19,305 DEBUG [RS:1;jenkins-hbase4:42469] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-08-17 16:56:19,305 DEBUG [RS:0;jenkins-hbase4:45379] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-08-17 16:56:19,306 DEBUG [RS:1;jenkins-hbase4:42469] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-08-17 16:56:19,306 DEBUG [RS:0;jenkins-hbase4:45379] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-08-17 16:56:19,306 DEBUG [RS:1;jenkins-hbase4:42469] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-17 16:56:19,306 DEBUG [RS:1;jenkins-hbase4:42469] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-17 16:56:19,307 DEBUG [RS:0;jenkins-hbase4:45379] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-17 16:56:19,307 DEBUG [RS:0;jenkins-hbase4:45379] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-17 16:56:19,307 DEBUG [RS:1;jenkins-hbase4:42469] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:19,307 DEBUG [RS:0;jenkins-hbase4:45379] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:19,307 DEBUG [RS:0;jenkins-hbase4:45379] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45379,1692291378053' 2023-08-17 16:56:19,307 DEBUG [RS:0;jenkins-hbase4:45379] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-08-17 16:56:19,307 DEBUG [RS:1;jenkins-hbase4:42469] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42469,1692291378095' 2023-08-17 16:56:19,307 DEBUG [RS:1;jenkins-hbase4:42469] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-08-17 16:56:19,308 DEBUG [RS:0;jenkins-hbase4:45379] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-08-17 16:56:19,308 DEBUG [RS:1;jenkins-hbase4:42469] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-08-17 16:56:19,308 DEBUG [RS:0;jenkins-hbase4:45379] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-17 16:56:19,308 INFO [RS:0;jenkins-hbase4:45379] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-17 16:56:19,308 DEBUG [RS:1;jenkins-hbase4:42469] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-17 16:56:19,308 INFO [RS:0;jenkins-hbase4:45379] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-17 16:56:19,308 INFO [RS:1;jenkins-hbase4:42469] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-17 16:56:19,309 INFO [RS:1;jenkins-hbase4:42469] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-17 16:56:19,368 DEBUG [jenkins-hbase4:33151] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-08-17 16:56:19,372 DEBUG [jenkins-hbase4:33151] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-17 16:56:19,378 DEBUG [jenkins-hbase4:33151] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-17 16:56:19,378 DEBUG [jenkins-hbase4:33151] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-17 16:56:19,378 DEBUG [jenkins-hbase4:33151] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-17 16:56:19,381 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45379,1692291378053, state=OPENING 2023-08-17 16:56:19,390 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-17 16:56:19,393 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:19,393 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-17 16:56:19,397 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45379,1692291378053}] 2023-08-17 16:56:19,416 INFO [RS:0;jenkins-hbase4:45379] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:19,416 INFO [RS:1;jenkins-hbase4:42469] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:19,420 INFO [RS:1;jenkins-hbase4:42469] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0, suffix=, logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095, archiveDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs, maxLogs=10 2023-08-17 16:56:19,420 INFO [RS:0;jenkins-hbase4:45379] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0, suffix=, logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053, archiveDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs, maxLogs=10 2023-08-17 16:56:19,443 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK] 2023-08-17 16:56:19,444 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK] 2023-08-17 16:56:19,444 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK] 2023-08-17 16:56:19,445 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK] 2023-08-17 16:56:19,448 INFO [RS:0;jenkins-hbase4:45379] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0.1692291379424 2023-08-17 16:56:19,450 INFO [RS:1;jenkins-hbase4:42469] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424 2023-08-17 16:56:19,454 DEBUG [RS:0;jenkins-hbase4:45379] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK], DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK]] 2023-08-17 16:56:19,454 DEBUG [RS:1;jenkins-hbase4:42469] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK], DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK]] 2023-08-17 16:56:19,581 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:19,584 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:19,586 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48420, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-17 16:56:19,599 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-17 16:56:19,600 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:19,600 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:19,600 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:19,604 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45379%2C1692291378053.meta, suffix=.meta, logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053, archiveDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs, maxLogs=10 2023-08-17 16:56:19,622 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK] 2023-08-17 16:56:19,622 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK] 2023-08-17 16:56:19,628 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.meta.1692291379605.meta 2023-08-17 16:56:19,632 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK], DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK]] 2023-08-17 16:56:19,632 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:19,634 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-17 16:56:19,653 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-17 16:56:19,676 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-17 16:56:19,684 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-17 16:56:19,684 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:19,684 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-17 16:56:19,684 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-17 16:56:19,687 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-17 16:56:19,689 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/info 2023-08-17 16:56:19,689 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/info 2023-08-17 16:56:19,690 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-17 16:56:19,691 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:19,691 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-17 16:56:19,693 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:56:19,693 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:56:19,693 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-17 16:56:19,694 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:19,694 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-17 16:56:19,696 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/table 2023-08-17 16:56:19,696 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/table 2023-08-17 16:56:19,696 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-17 16:56:19,697 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:19,699 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740 2023-08-17 16:56:19,701 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740 2023-08-17 16:56:19,705 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-17 16:56:19,707 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-17 16:56:19,709 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66592000, jitterRate=-0.007701873779296875}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-17 16:56:19,709 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-17 16:56:19,722 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692291379573 2023-08-17 16:56:19,739 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-17 16:56:19,740 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-17 16:56:19,741 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45379,1692291378053, state=OPEN 2023-08-17 16:56:19,744 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-08-17 16:56:19,744 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-17 16:56:19,748 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-17 16:56:19,748 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45379,1692291378053 in 347 msec 2023-08-17 16:56:19,753 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-17 16:56:19,753 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 555 msec 2023-08-17 16:56:19,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 773 msec 2023-08-17 16:56:19,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692291379758, completionTime=-1 2023-08-17 16:56:19,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-08-17 16:56:19,759 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-17 16:56:19,823 DEBUG [hconnection-0x557a2ee4-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:19,826 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48434, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:19,841 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-08-17 16:56:19,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692291439842 2023-08-17 16:56:19,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692291499842 2023-08-17 16:56:19,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 83 msec 2023-08-17 16:56:19,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33151,1692291376904-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33151,1692291376904-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33151,1692291376904-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33151, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,867 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:19,872 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-17 16:56:19,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-17 16:56:19,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:19,891 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-17 16:56:19,893 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-17 16:56:19,895 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-17 16:56:19,917 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:19,919 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2 empty. 2023-08-17 16:56:19,920 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:19,920 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-17 16:56:19,946 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-17 16:56:19,949 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => e13cf09e05e627f75f15f5ddbd0b03a2, NAME => 'hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp 2023-08-17 16:56:19,964 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:19,964 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing e13cf09e05e627f75f15f5ddbd0b03a2, disabling compactions & flushes 2023-08-17 16:56:19,964 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:19,964 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:19,964 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. after waiting 0 ms 2023-08-17 16:56:19,964 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:19,964 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:19,964 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for e13cf09e05e627f75f15f5ddbd0b03a2: 2023-08-17 16:56:19,968 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-17 16:56:19,982 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291379971"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692291379971"}]},"ts":"1692291379971"} 2023-08-17 16:56:20,012 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-17 16:56:20,013 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-17 16:56:20,018 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291380014"}]},"ts":"1692291380014"} 2023-08-17 16:56:20,021 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-17 16:56:20,026 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-17 16:56:20,028 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-17 16:56:20,028 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-17 16:56:20,028 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-17 16:56:20,030 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e13cf09e05e627f75f15f5ddbd0b03a2, ASSIGN}] 2023-08-17 16:56:20,033 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e13cf09e05e627f75f15f5ddbd0b03a2, ASSIGN 2023-08-17 16:56:20,034 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=e13cf09e05e627f75f15f5ddbd0b03a2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42469,1692291378095; forceNewPlan=false, retain=false 2023-08-17 16:56:20,187 INFO [jenkins-hbase4:33151] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-17 16:56:20,188 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e13cf09e05e627f75f15f5ddbd0b03a2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:20,189 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291380188"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291380188"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291380188"}]},"ts":"1692291380188"} 2023-08-17 16:56:20,195 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure e13cf09e05e627f75f15f5ddbd0b03a2, server=jenkins-hbase4.apache.org,42469,1692291378095}] 2023-08-17 16:56:20,349 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:20,349 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:20,352 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42698, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-17 16:56:20,357 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:20,357 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:20,360 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1, suffix=, logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095, archiveDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs, maxLogs=10 2023-08-17 16:56:20,377 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK] 2023-08-17 16:56:20,377 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK] 2023-08-17 16:56:20,382 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1.1692291380362 2023-08-17 16:56:20,383 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK], DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK]] 2023-08-17 16:56:20,383 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e13cf09e05e627f75f15f5ddbd0b03a2, NAME => 'hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:20,384 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:20,384 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:20,384 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:20,384 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:20,386 INFO [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:20,388 DEBUG [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/info 2023-08-17 16:56:20,388 DEBUG [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/info 2023-08-17 16:56:20,389 INFO [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e13cf09e05e627f75f15f5ddbd0b03a2 columnFamilyName info 2023-08-17 16:56:20,390 INFO [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] regionserver.HStore(310): Store=e13cf09e05e627f75f15f5ddbd0b03a2/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:20,392 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:20,392 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:20,397 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:20,401 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-17 16:56:20,402 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e13cf09e05e627f75f15f5ddbd0b03a2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63503408, jitterRate=-0.053725481033325195}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-17 16:56:20,402 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e13cf09e05e627f75f15f5ddbd0b03a2: 2023-08-17 16:56:20,406 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2., pid=6, masterSystemTime=1692291380349 2023-08-17 16:56:20,410 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:20,410 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:20,412 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e13cf09e05e627f75f15f5ddbd0b03a2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:20,412 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291380411"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291380411"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291380411"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291380411"}]},"ts":"1692291380411"} 2023-08-17 16:56:20,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-17 16:56:20,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure e13cf09e05e627f75f15f5ddbd0b03a2, server=jenkins-hbase4.apache.org,42469,1692291378095 in 221 msec 2023-08-17 16:56:20,423 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-17 16:56:20,424 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=e13cf09e05e627f75f15f5ddbd0b03a2, ASSIGN in 390 msec 2023-08-17 16:56:20,425 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-17 16:56:20,425 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291380425"}]},"ts":"1692291380425"} 2023-08-17 16:56:20,428 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-17 16:56:20,431 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-17 16:56:20,433 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 549 msec 2023-08-17 16:56:20,493 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-08-17 16:56:20,495 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-08-17 16:56:20,495 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:20,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:20,518 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42706, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:20,533 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-17 16:56:20,551 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-08-17 16:56:20,557 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 31 msec 2023-08-17 16:56:20,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-17 16:56:20,577 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-08-17 16:56:20,582 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-08-17 16:56:20,593 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-08-17 16:56:20,595 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-08-17 16:56:20,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.447sec 2023-08-17 16:56:20,598 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-17 16:56:20,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-17 16:56:20,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-17 16:56:20,601 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33151,1692291376904-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-17 16:56:20,601 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33151,1692291376904-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-17 16:56:20,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-17 16:56:20,620 DEBUG [Listener at localhost/44847] zookeeper.ReadOnlyZKClient(139): Connect 0x185830dc to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:20,626 DEBUG [Listener at localhost/44847] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@512eb72d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:20,639 DEBUG [hconnection-0x52e373f3-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:20,649 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48442, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:20,659 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33151,1692291376904 2023-08-17 16:56:20,659 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-17 16:56:20,659 INFO [Listener at localhost/44847] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/cluster_8b57984d-197b-73e4-852f-61f49906a62d, deleteOnExit=true 2023-08-17 16:56:20,659 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-17 16:56:20,670 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/test.cache.data in system properties and HBase conf 2023-08-17 16:56:20,670 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/hadoop.tmp.dir in system properties and HBase conf 2023-08-17 16:56:20,671 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/hadoop.log.dir in system properties and HBase conf 2023-08-17 16:56:20,671 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-17 16:56:20,671 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-17 16:56:20,671 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-17 16:56:20,671 DEBUG [Listener at localhost/44847] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-17 16:56:20,671 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-17 16:56:20,672 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-17 16:56:20,672 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-17 16:56:20,672 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-17 16:56:20,672 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-17 16:56:20,672 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-17 16:56:20,672 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-17 16:56:20,673 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-17 16:56:20,673 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-17 16:56:20,673 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/nfs.dump.dir in system properties and HBase conf 2023-08-17 16:56:20,673 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/java.io.tmpdir in system properties and HBase conf 2023-08-17 16:56:20,673 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-17 16:56:20,673 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-17 16:56:20,673 INFO [Listener at localhost/44847] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-17 16:56:20,679 WARN [Listener at localhost/44847] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-17 16:56:20,679 WARN [Listener at localhost/44847] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-17 16:56:20,722 WARN [Listener at localhost/44847] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-17 16:56:20,725 INFO [Listener at localhost/44847] log.Slf4jLog(67): jetty-6.1.26 2023-08-17 16:56:20,730 INFO [Listener at localhost/44847] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/java.io.tmpdir/Jetty_localhost_39155_hdfs____f94kd1/webapp 2023-08-17 16:56:20,822 INFO [Listener at localhost/44847] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39155 2023-08-17 16:56:20,875 WARN [Listener at localhost/44847] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-17 16:56:20,876 WARN [Listener at localhost/44847] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-17 16:56:20,926 WARN [Listener at localhost/41465] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-17 16:56:20,942 WARN [Listener at localhost/41465] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-17 16:56:20,945 WARN [Listener at localhost/41465] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-17 16:56:20,946 INFO [Listener at localhost/41465] log.Slf4jLog(67): jetty-6.1.26 2023-08-17 16:56:20,950 INFO [Listener at localhost/41465] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/java.io.tmpdir/Jetty_localhost_39515_datanode____6t1e8q/webapp 2023-08-17 16:56:21,042 INFO [Listener at localhost/41465] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39515 2023-08-17 16:56:21,051 WARN [Listener at localhost/44293] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-17 16:56:21,074 WARN [Listener at localhost/44293] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-17 16:56:21,076 WARN [Listener at localhost/44293] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-17 16:56:21,077 INFO [Listener at localhost/44293] log.Slf4jLog(67): jetty-6.1.26 2023-08-17 16:56:21,081 INFO [Listener at localhost/44293] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/java.io.tmpdir/Jetty_localhost_35233_datanode____tsov2v/webapp 2023-08-17 16:56:21,168 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd40f74b1b7257257: Processing first storage report for DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d from datanode f04c7764-baa1-403e-aa86-0841390ce624 2023-08-17 16:56:21,168 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd40f74b1b7257257: from storage DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d node DatanodeRegistration(127.0.0.1:35599, datanodeUuid=f04c7764-baa1-403e-aa86-0841390ce624, infoPort=34945, infoSecurePort=0, ipcPort=44293, storageInfo=lv=-57;cid=testClusterID;nsid=824962448;c=1692291380682), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-17 16:56:21,168 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd40f74b1b7257257: Processing first storage report for DS-bf3d497f-cde0-4f5e-b427-e53b5c8c4241 from datanode f04c7764-baa1-403e-aa86-0841390ce624 2023-08-17 16:56:21,168 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd40f74b1b7257257: from storage DS-bf3d497f-cde0-4f5e-b427-e53b5c8c4241 node DatanodeRegistration(127.0.0.1:35599, datanodeUuid=f04c7764-baa1-403e-aa86-0841390ce624, infoPort=34945, infoSecurePort=0, ipcPort=44293, storageInfo=lv=-57;cid=testClusterID;nsid=824962448;c=1692291380682), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-17 16:56:21,189 INFO [Listener at localhost/44293] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35233 2023-08-17 16:56:21,198 WARN [Listener at localhost/33349] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-17 16:56:21,222 WARN [Listener at localhost/33349] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-17 16:56:21,225 WARN [Listener at localhost/33349] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-17 16:56:21,226 INFO [Listener at localhost/33349] log.Slf4jLog(67): jetty-6.1.26 2023-08-17 16:56:21,229 INFO [Listener at localhost/33349] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/java.io.tmpdir/Jetty_localhost_37625_datanode____.urlfgk/webapp 2023-08-17 16:56:21,314 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb8f09b8fe663ee84: Processing first storage report for DS-f4a589e1-b6f6-42d8-9012-b432d943742e from datanode fa6903f1-cccf-4423-8dd2-e495823c58d9 2023-08-17 16:56:21,314 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb8f09b8fe663ee84: from storage DS-f4a589e1-b6f6-42d8-9012-b432d943742e node DatanodeRegistration(127.0.0.1:41833, datanodeUuid=fa6903f1-cccf-4423-8dd2-e495823c58d9, infoPort=41557, infoSecurePort=0, ipcPort=33349, storageInfo=lv=-57;cid=testClusterID;nsid=824962448;c=1692291380682), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-17 16:56:21,314 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb8f09b8fe663ee84: Processing first storage report for DS-36415ac6-f319-4ec9-a0ae-5c8c7dea4fd0 from datanode fa6903f1-cccf-4423-8dd2-e495823c58d9 2023-08-17 16:56:21,314 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb8f09b8fe663ee84: from storage DS-36415ac6-f319-4ec9-a0ae-5c8c7dea4fd0 node DatanodeRegistration(127.0.0.1:41833, datanodeUuid=fa6903f1-cccf-4423-8dd2-e495823c58d9, infoPort=41557, infoSecurePort=0, ipcPort=33349, storageInfo=lv=-57;cid=testClusterID;nsid=824962448;c=1692291380682), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-17 16:56:21,341 INFO [Listener at localhost/33349] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37625 2023-08-17 16:56:21,349 WARN [Listener at localhost/40177] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-17 16:56:21,370 WARN [Listener at localhost/40177] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-17 16:56:21,372 WARN [Listener at localhost/40177] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-17 16:56:21,373 INFO [Listener at localhost/40177] log.Slf4jLog(67): jetty-6.1.26 2023-08-17 16:56:21,378 INFO [Listener at localhost/40177] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/java.io.tmpdir/Jetty_localhost_35655_datanode____w2g1az/webapp 2023-08-17 16:56:21,465 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x56bf3db6476d7368: Processing first storage report for DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7 from datanode 240f4026-adbd-43ba-9d5b-6ae01f75beb8 2023-08-17 16:56:21,466 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x56bf3db6476d7368: from storage DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7 node DatanodeRegistration(127.0.0.1:33417, datanodeUuid=240f4026-adbd-43ba-9d5b-6ae01f75beb8, infoPort=43783, infoSecurePort=0, ipcPort=40177, storageInfo=lv=-57;cid=testClusterID;nsid=824962448;c=1692291380682), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-17 16:56:21,466 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x56bf3db6476d7368: Processing first storage report for DS-2c13bc2c-54fc-4d3f-9ba1-02a49c4cebfe from datanode 240f4026-adbd-43ba-9d5b-6ae01f75beb8 2023-08-17 16:56:21,466 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x56bf3db6476d7368: from storage DS-2c13bc2c-54fc-4d3f-9ba1-02a49c4cebfe node DatanodeRegistration(127.0.0.1:33417, datanodeUuid=240f4026-adbd-43ba-9d5b-6ae01f75beb8, infoPort=43783, infoSecurePort=0, ipcPort=40177, storageInfo=lv=-57;cid=testClusterID;nsid=824962448;c=1692291380682), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-17 16:56:21,485 INFO [Listener at localhost/40177] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35655 2023-08-17 16:56:21,494 WARN [Listener at localhost/35639] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-17 16:56:21,590 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xba06e40b03a94e9d: Processing first storage report for DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2 from datanode 04fbd196-147c-4b53-9a23-f2beaf7b81e0 2023-08-17 16:56:21,590 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xba06e40b03a94e9d: from storage DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2 node DatanodeRegistration(127.0.0.1:32949, datanodeUuid=04fbd196-147c-4b53-9a23-f2beaf7b81e0, infoPort=34939, infoSecurePort=0, ipcPort=35639, storageInfo=lv=-57;cid=testClusterID;nsid=824962448;c=1692291380682), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-17 16:56:21,590 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xba06e40b03a94e9d: Processing first storage report for DS-f3824fda-8448-4271-a4a6-229551690d7b from datanode 04fbd196-147c-4b53-9a23-f2beaf7b81e0 2023-08-17 16:56:21,590 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xba06e40b03a94e9d: from storage DS-f3824fda-8448-4271-a4a6-229551690d7b node DatanodeRegistration(127.0.0.1:32949, datanodeUuid=04fbd196-147c-4b53-9a23-f2beaf7b81e0, infoPort=34939, infoSecurePort=0, ipcPort=35639, storageInfo=lv=-57;cid=testClusterID;nsid=824962448;c=1692291380682), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-17 16:56:21,604 DEBUG [Listener at localhost/35639] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8 2023-08-17 16:56:21,604 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,606 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,638 INFO [Listener at localhost/35639] util.FSUtils(471): Created version file at hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4 with version=8 2023-08-17 16:56:21,638 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/hbase-staging 2023-08-17 16:56:21,640 INFO [Listener at localhost/35639] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-08-17 16:56:21,640 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,641 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,641 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:21,641 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,641 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:21,641 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:21,645 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34345 2023-08-17 16:56:21,645 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,647 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,647 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34345 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:21,650 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:343450x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:21,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:34345-0x1001b2c4f410007 connected 2023-08-17 16:56:21,663 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:56:21,663 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:21,664 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-17 16:56:21,664 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34345 2023-08-17 16:56:21,664 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34345 2023-08-17 16:56:21,665 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34345 2023-08-17 16:56:21,665 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34345 2023-08-17 16:56:21,665 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34345 2023-08-17 16:56:21,665 INFO [Listener at localhost/35639] master.HMaster(444): hbase.rootdir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4, hbase.cluster.distributed=false 2023-08-17 16:56:21,679 INFO [Listener at localhost/35639] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-17 16:56:21,679 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,679 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,679 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:21,680 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,680 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:21,680 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:21,681 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38799 2023-08-17 16:56:21,682 INFO [Listener at localhost/35639] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-17 16:56:21,683 DEBUG [Listener at localhost/35639] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-17 16:56:21,683 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,685 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,686 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38799 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:21,689 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:387990x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:21,691 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38799-0x1001b2c4f410008 connected 2023-08-17 16:56:21,691 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:56:21,691 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:21,692 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-17 16:56:21,693 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38799 2023-08-17 16:56:21,694 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38799 2023-08-17 16:56:21,694 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38799 2023-08-17 16:56:21,694 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38799 2023-08-17 16:56:21,695 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38799 2023-08-17 16:56:21,706 INFO [Listener at localhost/35639] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-17 16:56:21,706 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,706 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,706 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:21,706 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,706 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:21,707 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:21,708 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38843 2023-08-17 16:56:21,708 INFO [Listener at localhost/35639] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-17 16:56:21,709 DEBUG [Listener at localhost/35639] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-17 16:56:21,710 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,711 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,712 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38843 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:21,716 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:388430x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:21,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38843-0x1001b2c4f410009 connected 2023-08-17 16:56:21,718 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:56:21,718 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:21,719 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-17 16:56:21,719 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38843 2023-08-17 16:56:21,720 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38843 2023-08-17 16:56:21,720 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38843 2023-08-17 16:56:21,720 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38843 2023-08-17 16:56:21,720 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38843 2023-08-17 16:56:21,734 INFO [Listener at localhost/35639] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-17 16:56:21,734 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,734 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,735 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:21,735 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,735 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:21,735 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:21,737 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33525 2023-08-17 16:56:21,737 INFO [Listener at localhost/35639] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-17 16:56:21,738 DEBUG [Listener at localhost/35639] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-17 16:56:21,738 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,739 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,740 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33525 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:21,743 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:335250x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:21,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33525-0x1001b2c4f41000a connected 2023-08-17 16:56:21,745 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:56:21,745 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:21,746 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-17 16:56:21,746 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33525 2023-08-17 16:56:21,746 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33525 2023-08-17 16:56:21,747 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33525 2023-08-17 16:56:21,747 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33525 2023-08-17 16:56:21,747 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33525 2023-08-17 16:56:21,758 INFO [Listener at localhost/35639] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-17 16:56:21,758 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,758 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,758 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:21,759 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:21,759 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:21,759 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:21,760 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41501 2023-08-17 16:56:21,761 INFO [Listener at localhost/35639] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-17 16:56:21,762 DEBUG [Listener at localhost/35639] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-17 16:56:21,762 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,763 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,764 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41501 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:21,767 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:415010x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:21,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41501-0x1001b2c4f41000b connected 2023-08-17 16:56:21,769 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:56:21,769 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:21,770 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-17 16:56:21,770 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41501 2023-08-17 16:56:21,770 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41501 2023-08-17 16:56:21,771 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41501 2023-08-17 16:56:21,772 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41501 2023-08-17 16:56:21,773 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41501 2023-08-17 16:56:21,774 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,34345,1692291381639 2023-08-17 16:56:21,776 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-17 16:56:21,776 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,34345,1692291381639 2023-08-17 16:56:21,779 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-17 16:56:21,779 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-17 16:56:21,779 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-17 16:56:21,779 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-17 16:56:21,779 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-17 16:56:21,780 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:21,782 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-17 16:56:21,784 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,34345,1692291381639 from backup master directory 2023-08-17 16:56:21,784 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-17 16:56:21,786 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,34345,1692291381639 2023-08-17 16:56:21,786 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-17 16:56:21,786 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:21,786 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,34345,1692291381639 2023-08-17 16:56:21,815 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/hbase.id with ID: 54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:21,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:21,834 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:21,856 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2190a892 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:21,860 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4fb44140, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:21,860 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:21,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-17 16:56:21,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:21,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:21,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store-tmp 2023-08-17 16:56:21,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:21,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-17 16:56:21,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:21,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:21,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-17 16:56:21,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:21,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:21,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:56:21,880 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34345,1692291381639 2023-08-17 16:56:21,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:21,884 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34345%2C1692291381639.jenkins-hbase4.apache.org%2C34345%2C1692291381639.regiongroup-0, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34345,1692291381639, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/oldWALs, maxLogs=10 2023-08-17 16:56:21,903 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK] 2023-08-17 16:56:21,907 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:21,907 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:56:21,913 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34345,1692291381639/jenkins-hbase4.apache.org%2C34345%2C1692291381639.jenkins-hbase4.apache.org%2C34345%2C1692291381639.regiongroup-0.1692291381884 2023-08-17 16:56:21,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK], DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK]] 2023-08-17 16:56:21,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:21,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:21,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:21,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:21,916 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:21,918 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-17 16:56:21,918 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-17 16:56:21,919 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:21,921 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:21,922 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:21,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:21,929 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-17 16:56:21,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71209778, jitterRate=0.06110838055610657}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-17 16:56:21,929 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:56:21,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-17 16:56:21,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-17 16:56:21,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-17 16:56:21,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-17 16:56:21,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-17 16:56:21,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-17 16:56:21,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-17 16:56:21,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-17 16:56:21,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-17 16:56:21,945 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-17 16:56:21,945 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-17 16:56:21,946 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-08-17 16:56:21,946 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-17 16:56:21,946 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-08-17 16:56:21,950 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:21,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-08-17 16:56:21,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-08-17 16:56:21,952 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-08-17 16:56:21,953 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-17 16:56:21,953 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-17 16:56:21,953 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-17 16:56:21,953 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-17 16:56:21,953 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-17 16:56:21,954 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:21,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,34345,1692291381639, sessionid=0x1001b2c4f410007, setting cluster-up flag (Was=false) 2023-08-17 16:56:21,959 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:21,967 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-08-17 16:56:21,968 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34345,1692291381639 2023-08-17 16:56:21,972 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:21,979 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-08-17 16:56:21,980 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34345,1692291381639 2023-08-17 16:56:21,980 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.hbase-snapshot/.tmp 2023-08-17 16:56:21,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-17 16:56:21,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:21,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:21,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:21,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:21,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-17 16:56:21,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:21,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:21,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:21,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692291411985 2023-08-17 16:56:21,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-17 16:56:21,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-17 16:56:21,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-17 16:56:21,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-17 16:56:21,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-17 16:56:21,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-17 16:56:21,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:21,987 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-17 16:56:21,987 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-17 16:56:21,988 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-17 16:56:21,988 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-17 16:56:21,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-17 16:56:21,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-17 16:56:21,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-17 16:56:21,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291381989,5,FailOnTimeoutGroup] 2023-08-17 16:56:21,989 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:21,990 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291381989,5,FailOnTimeoutGroup] 2023-08-17 16:56:21,990 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:21,990 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-17 16:56:21,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:21,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,020 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-17 16:56:22,021 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-17 16:56:22,021 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4 2023-08-17 16:56:22,037 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:22,039 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-17 16:56:22,043 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info 2023-08-17 16:56:22,043 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-17 16:56:22,044 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:22,044 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-17 16:56:22,046 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:56:22,046 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-17 16:56:22,047 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:22,047 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-17 16:56:22,048 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table 2023-08-17 16:56:22,049 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-17 16:56:22,049 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:22,051 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740 2023-08-17 16:56:22,052 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740 2023-08-17 16:56:22,055 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-17 16:56:22,057 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-17 16:56:22,061 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-17 16:56:22,063 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69818775, jitterRate=0.040380820631980896}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-17 16:56:22,063 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-17 16:56:22,063 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-17 16:56:22,063 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-17 16:56:22,063 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-17 16:56:22,063 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-17 16:56:22,063 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-17 16:56:22,063 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-17 16:56:22,064 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-17 16:56:22,066 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-17 16:56:22,066 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-17 16:56:22,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-17 16:56:22,069 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-17 16:56:22,071 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-17 16:56:22,077 INFO [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(951): ClusterId : 54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:22,077 INFO [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(951): ClusterId : 54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:22,078 DEBUG [RS:0;jenkins-hbase4:38799] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-17 16:56:22,079 DEBUG [RS:1;jenkins-hbase4:38843] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-17 16:56:22,077 INFO [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(951): ClusterId : 54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:22,077 INFO [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(951): ClusterId : 54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:22,080 DEBUG [RS:2;jenkins-hbase4:33525] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-17 16:56:22,081 DEBUG [RS:3;jenkins-hbase4:41501] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-17 16:56:22,083 DEBUG [RS:1;jenkins-hbase4:38843] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-17 16:56:22,083 DEBUG [RS:1;jenkins-hbase4:38843] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-17 16:56:22,083 DEBUG [RS:0;jenkins-hbase4:38799] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-17 16:56:22,084 DEBUG [RS:0;jenkins-hbase4:38799] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-17 16:56:22,084 DEBUG [RS:2;jenkins-hbase4:33525] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-17 16:56:22,085 DEBUG [RS:2;jenkins-hbase4:33525] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-17 16:56:22,087 DEBUG [RS:3;jenkins-hbase4:41501] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-17 16:56:22,087 DEBUG [RS:3;jenkins-hbase4:41501] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-17 16:56:22,088 DEBUG [RS:1;jenkins-hbase4:38843] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-17 16:56:22,089 DEBUG [RS:1;jenkins-hbase4:38843] zookeeper.ReadOnlyZKClient(139): Connect 0x4c597470 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:22,089 DEBUG [RS:0;jenkins-hbase4:38799] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-17 16:56:22,089 DEBUG [RS:2;jenkins-hbase4:33525] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-17 16:56:22,092 DEBUG [RS:0;jenkins-hbase4:38799] zookeeper.ReadOnlyZKClient(139): Connect 0x6dcdd537 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:22,092 DEBUG [RS:2;jenkins-hbase4:33525] zookeeper.ReadOnlyZKClient(139): Connect 0x704bb248 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:22,092 DEBUG [RS:3;jenkins-hbase4:41501] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-17 16:56:22,094 DEBUG [RS:3;jenkins-hbase4:41501] zookeeper.ReadOnlyZKClient(139): Connect 0x2636e685 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:22,103 DEBUG [RS:1;jenkins-hbase4:38843] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@429e838b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:22,104 DEBUG [RS:1;jenkins-hbase4:38843] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a389b0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:56:22,104 DEBUG [RS:0;jenkins-hbase4:38799] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@630a36fe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:22,104 DEBUG [RS:0;jenkins-hbase4:38799] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6735be47, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:56:22,106 DEBUG [RS:2;jenkins-hbase4:33525] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@74742899, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:22,106 DEBUG [RS:2;jenkins-hbase4:33525] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3176b873, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:56:22,106 DEBUG [RS:3;jenkins-hbase4:41501] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@28f10c33, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:22,106 DEBUG [RS:3;jenkins-hbase4:41501] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2cce8df5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:56:22,113 DEBUG [RS:1;jenkins-hbase4:38843] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:38843 2023-08-17 16:56:22,113 INFO [RS:1;jenkins-hbase4:38843] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-17 16:56:22,113 INFO [RS:1;jenkins-hbase4:38843] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-17 16:56:22,113 DEBUG [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1022): About to register with Master. 2023-08-17 16:56:22,114 INFO [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34345,1692291381639 with isa=jenkins-hbase4.apache.org/172.31.14.131:38843, startcode=1692291381706 2023-08-17 16:56:22,114 DEBUG [RS:1;jenkins-hbase4:38843] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-17 16:56:22,114 DEBUG [RS:3;jenkins-hbase4:41501] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:41501 2023-08-17 16:56:22,114 INFO [RS:3;jenkins-hbase4:41501] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-17 16:56:22,114 INFO [RS:3;jenkins-hbase4:41501] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-17 16:56:22,114 DEBUG [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(1022): About to register with Master. 2023-08-17 16:56:22,115 DEBUG [RS:0;jenkins-hbase4:38799] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:38799 2023-08-17 16:56:22,116 INFO [RS:0;jenkins-hbase4:38799] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-17 16:56:22,116 INFO [RS:0;jenkins-hbase4:38799] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-17 16:56:22,116 INFO [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34345,1692291381639 with isa=jenkins-hbase4.apache.org/172.31.14.131:41501, startcode=1692291381758 2023-08-17 16:56:22,116 DEBUG [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1022): About to register with Master. 2023-08-17 16:56:22,116 DEBUG [RS:3;jenkins-hbase4:41501] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-17 16:56:22,116 INFO [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34345,1692291381639 with isa=jenkins-hbase4.apache.org/172.31.14.131:38799, startcode=1692291381679 2023-08-17 16:56:22,117 DEBUG [RS:0;jenkins-hbase4:38799] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-17 16:56:22,117 DEBUG [RS:2;jenkins-hbase4:33525] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:33525 2023-08-17 16:56:22,117 INFO [RS:2;jenkins-hbase4:33525] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-17 16:56:22,117 INFO [RS:2;jenkins-hbase4:33525] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-17 16:56:22,117 DEBUG [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(1022): About to register with Master. 2023-08-17 16:56:22,118 INFO [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34345,1692291381639 with isa=jenkins-hbase4.apache.org/172.31.14.131:33525, startcode=1692291381734 2023-08-17 16:56:22,118 DEBUG [RS:2;jenkins-hbase4:33525] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-17 16:56:22,118 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43873, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-08-17 16:56:22,122 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34345] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,125 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40847, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-08-17 16:56:22,125 DEBUG [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4 2023-08-17 16:56:22,125 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38859, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-08-17 16:56:22,125 DEBUG [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41465 2023-08-17 16:56:22,125 DEBUG [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-17 16:56:22,125 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34345] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:22,125 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40967, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-08-17 16:56:22,126 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34345] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:22,126 DEBUG [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4 2023-08-17 16:56:22,126 DEBUG [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41465 2023-08-17 16:56:22,126 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34345] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:22,126 DEBUG [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-17 16:56:22,126 DEBUG [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4 2023-08-17 16:56:22,126 DEBUG [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41465 2023-08-17 16:56:22,126 DEBUG [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-17 16:56:22,127 DEBUG [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4 2023-08-17 16:56:22,127 DEBUG [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41465 2023-08-17 16:56:22,127 DEBUG [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-17 16:56:22,127 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:22,131 DEBUG [RS:0;jenkins-hbase4:38799] zookeeper.ZKUtil(162): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,131 DEBUG [RS:3;jenkins-hbase4:41501] zookeeper.ZKUtil(162): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:22,131 WARN [RS:0;jenkins-hbase4:38799] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:22,131 DEBUG [RS:1;jenkins-hbase4:38843] zookeeper.ZKUtil(162): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:22,131 DEBUG [RS:2;jenkins-hbase4:33525] zookeeper.ZKUtil(162): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:22,131 WARN [RS:1;jenkins-hbase4:38843] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:22,131 INFO [RS:0;jenkins-hbase4:38799] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:22,131 INFO [RS:1;jenkins-hbase4:38843] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:22,131 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38799,1692291381679] 2023-08-17 16:56:22,131 WARN [RS:3;jenkins-hbase4:41501] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:22,131 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38843,1692291381706] 2023-08-17 16:56:22,131 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33525,1692291381734] 2023-08-17 16:56:22,131 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41501,1692291381758] 2023-08-17 16:56:22,131 INFO [RS:1;jenkins-hbase4:38843] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:22,131 INFO [RS:0;jenkins-hbase4:38799] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:22,131 WARN [RS:2;jenkins-hbase4:33525] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:22,132 DEBUG [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:22,132 DEBUG [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,132 INFO [RS:2;jenkins-hbase4:33525] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:22,131 INFO [RS:3;jenkins-hbase4:41501] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:22,132 INFO [RS:2;jenkins-hbase4:33525] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:22,132 INFO [RS:3;jenkins-hbase4:41501] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:22,132 DEBUG [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:22,132 DEBUG [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:22,142 DEBUG [RS:0;jenkins-hbase4:38799] zookeeper.ZKUtil(162): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:22,142 DEBUG [RS:2;jenkins-hbase4:33525] zookeeper.ZKUtil(162): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:22,142 DEBUG [RS:1;jenkins-hbase4:38843] zookeeper.ZKUtil(162): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:22,142 DEBUG [RS:2;jenkins-hbase4:33525] zookeeper.ZKUtil(162): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,142 DEBUG [RS:0;jenkins-hbase4:38799] zookeeper.ZKUtil(162): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,142 DEBUG [RS:1;jenkins-hbase4:38843] zookeeper.ZKUtil(162): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,143 DEBUG [RS:2;jenkins-hbase4:33525] zookeeper.ZKUtil(162): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:22,143 DEBUG [RS:1;jenkins-hbase4:38843] zookeeper.ZKUtil(162): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:22,143 DEBUG [RS:0;jenkins-hbase4:38799] zookeeper.ZKUtil(162): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:22,144 DEBUG [RS:2;jenkins-hbase4:33525] zookeeper.ZKUtil(162): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:22,144 DEBUG [RS:1;jenkins-hbase4:38843] zookeeper.ZKUtil(162): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:22,144 DEBUG [RS:0;jenkins-hbase4:38799] zookeeper.ZKUtil(162): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:22,145 DEBUG [RS:2;jenkins-hbase4:33525] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:56:22,145 DEBUG [RS:1;jenkins-hbase4:38843] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:56:22,145 DEBUG [RS:0;jenkins-hbase4:38799] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:56:22,145 INFO [RS:1;jenkins-hbase4:38843] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-17 16:56:22,145 INFO [RS:2;jenkins-hbase4:33525] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-17 16:56:22,146 INFO [RS:0;jenkins-hbase4:38799] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-17 16:56:22,149 DEBUG [RS:3;jenkins-hbase4:41501] zookeeper.ZKUtil(162): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:22,149 DEBUG [RS:3;jenkins-hbase4:41501] zookeeper.ZKUtil(162): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,150 DEBUG [RS:3;jenkins-hbase4:41501] zookeeper.ZKUtil(162): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:22,150 DEBUG [RS:3;jenkins-hbase4:41501] zookeeper.ZKUtil(162): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:22,152 DEBUG [RS:3;jenkins-hbase4:41501] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:56:22,152 INFO [RS:3;jenkins-hbase4:41501] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-17 16:56:22,152 INFO [RS:1;jenkins-hbase4:38843] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-17 16:56:22,153 INFO [RS:2;jenkins-hbase4:33525] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-17 16:56:22,157 INFO [RS:1;jenkins-hbase4:38843] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-17 16:56:22,157 INFO [RS:1;jenkins-hbase4:38843] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,161 INFO [RS:3;jenkins-hbase4:41501] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-17 16:56:22,161 INFO [RS:0;jenkins-hbase4:38799] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-17 16:56:22,161 INFO [RS:2;jenkins-hbase4:33525] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-17 16:56:22,161 INFO [RS:2;jenkins-hbase4:33525] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,161 INFO [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-17 16:56:22,163 INFO [RS:3;jenkins-hbase4:41501] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-17 16:56:22,163 INFO [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-17 16:56:22,163 INFO [RS:3;jenkins-hbase4:41501] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,164 INFO [RS:0;jenkins-hbase4:38799] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-17 16:56:22,164 INFO [RS:0;jenkins-hbase4:38799] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,165 INFO [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-17 16:56:22,166 INFO [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-17 16:56:22,166 INFO [RS:2;jenkins-hbase4:33525] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,166 INFO [RS:1;jenkins-hbase4:38843] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,166 DEBUG [RS:2;jenkins-hbase4:33525] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,166 DEBUG [RS:1;jenkins-hbase4:38843] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,166 DEBUG [RS:2;jenkins-hbase4:33525] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,166 DEBUG [RS:1;jenkins-hbase4:38843] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,166 DEBUG [RS:2;jenkins-hbase4:33525] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,166 DEBUG [RS:1;jenkins-hbase4:38843] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 DEBUG [RS:2;jenkins-hbase4:33525] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 DEBUG [RS:1;jenkins-hbase4:38843] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 INFO [RS:3;jenkins-hbase4:41501] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,167 DEBUG [RS:1;jenkins-hbase4:38843] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 INFO [RS:0;jenkins-hbase4:38799] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,167 DEBUG [RS:2;jenkins-hbase4:33525] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 DEBUG [RS:3;jenkins-hbase4:41501] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 DEBUG [RS:2;jenkins-hbase4:33525] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:22,167 DEBUG [RS:3;jenkins-hbase4:41501] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 DEBUG [RS:2;jenkins-hbase4:33525] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 DEBUG [RS:3;jenkins-hbase4:41501] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 DEBUG [RS:2;jenkins-hbase4:33525] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 DEBUG [RS:3;jenkins-hbase4:41501] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,167 DEBUG [RS:2;jenkins-hbase4:33525] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,168 DEBUG [RS:3;jenkins-hbase4:41501] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,168 DEBUG [RS:2;jenkins-hbase4:33525] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,168 DEBUG [RS:3;jenkins-hbase4:41501] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:22,167 DEBUG [RS:1;jenkins-hbase4:38843] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:22,168 DEBUG [RS:3;jenkins-hbase4:41501] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,168 DEBUG [RS:1;jenkins-hbase4:38843] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,168 DEBUG [RS:3;jenkins-hbase4:41501] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,168 INFO [RS:2;jenkins-hbase4:33525] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,167 DEBUG [RS:0;jenkins-hbase4:38799] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,168 INFO [RS:2;jenkins-hbase4:33525] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,168 DEBUG [RS:0;jenkins-hbase4:38799] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,168 INFO [RS:2;jenkins-hbase4:33525] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,168 DEBUG [RS:0;jenkins-hbase4:38799] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,168 DEBUG [RS:3;jenkins-hbase4:41501] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,168 DEBUG [RS:1;jenkins-hbase4:38843] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,169 DEBUG [RS:3;jenkins-hbase4:41501] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,169 DEBUG [RS:1;jenkins-hbase4:38843] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,169 DEBUG [RS:0;jenkins-hbase4:38799] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,169 DEBUG [RS:1;jenkins-hbase4:38843] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,169 DEBUG [RS:0;jenkins-hbase4:38799] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,169 DEBUG [RS:0;jenkins-hbase4:38799] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:22,169 DEBUG [RS:0;jenkins-hbase4:38799] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,169 DEBUG [RS:0;jenkins-hbase4:38799] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,169 DEBUG [RS:0;jenkins-hbase4:38799] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,170 DEBUG [RS:0;jenkins-hbase4:38799] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:22,181 INFO [RS:1;jenkins-hbase4:38843] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,181 INFO [RS:1;jenkins-hbase4:38843] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,181 INFO [RS:0;jenkins-hbase4:38799] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,181 INFO [RS:1;jenkins-hbase4:38843] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,181 INFO [RS:3;jenkins-hbase4:41501] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,181 INFO [RS:0;jenkins-hbase4:38799] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,181 INFO [RS:3;jenkins-hbase4:41501] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,181 INFO [RS:0;jenkins-hbase4:38799] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,181 INFO [RS:3;jenkins-hbase4:41501] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,190 INFO [RS:2;jenkins-hbase4:33525] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-17 16:56:22,190 INFO [RS:2;jenkins-hbase4:33525] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33525,1692291381734-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,195 INFO [RS:0;jenkins-hbase4:38799] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-17 16:56:22,195 INFO [RS:0;jenkins-hbase4:38799] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38799,1692291381679-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,195 INFO [RS:3;jenkins-hbase4:41501] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-17 16:56:22,196 INFO [RS:3;jenkins-hbase4:41501] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41501,1692291381758-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,197 INFO [RS:1;jenkins-hbase4:38843] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-17 16:56:22,197 INFO [RS:1;jenkins-hbase4:38843] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38843,1692291381706-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,201 INFO [RS:2;jenkins-hbase4:33525] regionserver.Replication(203): jenkins-hbase4.apache.org,33525,1692291381734 started 2023-08-17 16:56:22,202 INFO [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,33525,1692291381734, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33525, sessionid=0x1001b2c4f41000a 2023-08-17 16:56:22,202 DEBUG [RS:2;jenkins-hbase4:33525] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-17 16:56:22,202 DEBUG [RS:2;jenkins-hbase4:33525] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:22,202 DEBUG [RS:2;jenkins-hbase4:33525] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33525,1692291381734' 2023-08-17 16:56:22,202 DEBUG [RS:2;jenkins-hbase4:33525] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-17 16:56:22,202 DEBUG [RS:2;jenkins-hbase4:33525] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-17 16:56:22,203 DEBUG [RS:2;jenkins-hbase4:33525] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-17 16:56:22,203 DEBUG [RS:2;jenkins-hbase4:33525] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-17 16:56:22,203 DEBUG [RS:2;jenkins-hbase4:33525] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:22,204 DEBUG [RS:2;jenkins-hbase4:33525] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33525,1692291381734' 2023-08-17 16:56:22,204 DEBUG [RS:2;jenkins-hbase4:33525] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-17 16:56:22,204 DEBUG [RS:2;jenkins-hbase4:33525] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-17 16:56:22,204 DEBUG [RS:2;jenkins-hbase4:33525] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-17 16:56:22,204 INFO [RS:2;jenkins-hbase4:33525] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-17 16:56:22,204 INFO [RS:2;jenkins-hbase4:33525] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-17 16:56:22,207 INFO [RS:3;jenkins-hbase4:41501] regionserver.Replication(203): jenkins-hbase4.apache.org,41501,1692291381758 started 2023-08-17 16:56:22,207 INFO [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41501,1692291381758, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41501, sessionid=0x1001b2c4f41000b 2023-08-17 16:56:22,207 DEBUG [RS:3;jenkins-hbase4:41501] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-17 16:56:22,207 DEBUG [RS:3;jenkins-hbase4:41501] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:22,207 DEBUG [RS:3;jenkins-hbase4:41501] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41501,1692291381758' 2023-08-17 16:56:22,207 DEBUG [RS:3;jenkins-hbase4:41501] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-17 16:56:22,208 DEBUG [RS:3;jenkins-hbase4:41501] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-17 16:56:22,208 DEBUG [RS:3;jenkins-hbase4:41501] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-17 16:56:22,208 DEBUG [RS:3;jenkins-hbase4:41501] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-17 16:56:22,208 DEBUG [RS:3;jenkins-hbase4:41501] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:22,208 DEBUG [RS:3;jenkins-hbase4:41501] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41501,1692291381758' 2023-08-17 16:56:22,208 DEBUG [RS:3;jenkins-hbase4:41501] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-17 16:56:22,208 DEBUG [RS:3;jenkins-hbase4:41501] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-17 16:56:22,209 DEBUG [RS:3;jenkins-hbase4:41501] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-17 16:56:22,209 INFO [RS:3;jenkins-hbase4:41501] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-17 16:56:22,209 INFO [RS:3;jenkins-hbase4:41501] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-17 16:56:22,211 INFO [RS:0;jenkins-hbase4:38799] regionserver.Replication(203): jenkins-hbase4.apache.org,38799,1692291381679 started 2023-08-17 16:56:22,211 INFO [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38799,1692291381679, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38799, sessionid=0x1001b2c4f410008 2023-08-17 16:56:22,212 DEBUG [RS:0;jenkins-hbase4:38799] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-17 16:56:22,212 DEBUG [RS:0;jenkins-hbase4:38799] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,212 DEBUG [RS:0;jenkins-hbase4:38799] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38799,1692291381679' 2023-08-17 16:56:22,212 DEBUG [RS:0;jenkins-hbase4:38799] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-17 16:56:22,212 DEBUG [RS:0;jenkins-hbase4:38799] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-17 16:56:22,212 INFO [RS:1;jenkins-hbase4:38843] regionserver.Replication(203): jenkins-hbase4.apache.org,38843,1692291381706 started 2023-08-17 16:56:22,212 INFO [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38843,1692291381706, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38843, sessionid=0x1001b2c4f410009 2023-08-17 16:56:22,212 DEBUG [RS:1;jenkins-hbase4:38843] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-17 16:56:22,212 DEBUG [RS:1;jenkins-hbase4:38843] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:22,212 DEBUG [RS:1;jenkins-hbase4:38843] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38843,1692291381706' 2023-08-17 16:56:22,213 DEBUG [RS:1;jenkins-hbase4:38843] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-17 16:56:22,213 DEBUG [RS:0;jenkins-hbase4:38799] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-17 16:56:22,213 DEBUG [RS:0;jenkins-hbase4:38799] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-17 16:56:22,213 DEBUG [RS:0;jenkins-hbase4:38799] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,213 DEBUG [RS:0;jenkins-hbase4:38799] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38799,1692291381679' 2023-08-17 16:56:22,213 DEBUG [RS:0;jenkins-hbase4:38799] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-17 16:56:22,213 DEBUG [RS:1;jenkins-hbase4:38843] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-17 16:56:22,213 DEBUG [RS:0;jenkins-hbase4:38799] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-17 16:56:22,213 DEBUG [RS:1;jenkins-hbase4:38843] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-17 16:56:22,213 DEBUG [RS:1;jenkins-hbase4:38843] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-17 16:56:22,213 DEBUG [RS:1;jenkins-hbase4:38843] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:22,213 DEBUG [RS:1;jenkins-hbase4:38843] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38843,1692291381706' 2023-08-17 16:56:22,213 DEBUG [RS:0;jenkins-hbase4:38799] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-17 16:56:22,213 DEBUG [RS:1;jenkins-hbase4:38843] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-17 16:56:22,214 INFO [RS:0;jenkins-hbase4:38799] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-17 16:56:22,214 INFO [RS:0;jenkins-hbase4:38799] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-17 16:56:22,214 DEBUG [RS:1;jenkins-hbase4:38843] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-17 16:56:22,214 DEBUG [RS:1;jenkins-hbase4:38843] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-17 16:56:22,214 INFO [RS:1;jenkins-hbase4:38843] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-17 16:56:22,214 INFO [RS:1;jenkins-hbase4:38843] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-17 16:56:22,221 DEBUG [jenkins-hbase4:34345] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-08-17 16:56:22,221 DEBUG [jenkins-hbase4:34345] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-17 16:56:22,224 DEBUG [jenkins-hbase4:34345] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-17 16:56:22,224 DEBUG [jenkins-hbase4:34345] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-17 16:56:22,224 DEBUG [jenkins-hbase4:34345] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-17 16:56:22,224 DEBUG [jenkins-hbase4:34345] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-17 16:56:22,224 DEBUG [jenkins-hbase4:34345] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-17 16:56:22,225 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38843,1692291381706, state=OPENING 2023-08-17 16:56:22,226 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-17 16:56:22,227 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:22,228 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38843,1692291381706}] 2023-08-17 16:56:22,228 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-17 16:56:22,305 INFO [RS:2;jenkins-hbase4:33525] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:22,307 INFO [RS:2;jenkins-hbase4:33525] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33525%2C1692291381734.jenkins-hbase4.apache.org%2C33525%2C1692291381734.regiongroup-0, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,33525,1692291381734, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:56:22,309 INFO [RS:3;jenkins-hbase4:41501] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:22,311 INFO [RS:3;jenkins-hbase4:41501] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41501%2C1692291381758.jenkins-hbase4.apache.org%2C41501%2C1692291381758.regiongroup-0, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,41501,1692291381758, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:56:22,314 INFO [RS:0;jenkins-hbase4:38799] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:22,315 INFO [RS:1;jenkins-hbase4:38843] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:22,318 INFO [RS:0;jenkins-hbase4:38799] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38799%2C1692291381679.jenkins-hbase4.apache.org%2C38799%2C1692291381679.regiongroup-0, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38799,1692291381679, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:56:22,318 INFO [RS:1;jenkins-hbase4:38843] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38843%2C1692291381706.jenkins-hbase4.apache.org%2C38843%2C1692291381706.regiongroup-0, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:56:22,334 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK] 2023-08-17 16:56:22,335 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:56:22,334 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:56:22,351 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:56:22,351 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK] 2023-08-17 16:56:22,352 INFO [RS:2;jenkins-hbase4:33525] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,33525,1692291381734/jenkins-hbase4.apache.org%2C33525%2C1692291381734.jenkins-hbase4.apache.org%2C33525%2C1692291381734.regiongroup-0.1692291382309 2023-08-17 16:56:22,352 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:22,352 DEBUG [RS:2;jenkins-hbase4:33525] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK], DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK], DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK]] 2023-08-17 16:56:22,359 INFO [RS:3;jenkins-hbase4:41501] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,41501,1692291381758/jenkins-hbase4.apache.org%2C41501%2C1692291381758.jenkins-hbase4.apache.org%2C41501%2C1692291381758.regiongroup-0.1692291382312 2023-08-17 16:56:22,359 DEBUG [RS:3;jenkins-hbase4:41501] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK], DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK]] 2023-08-17 16:56:22,374 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:56:22,374 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:56:22,374 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:22,375 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:56:22,376 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:56:22,376 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:22,387 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:22,387 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:22,389 INFO [RS:0;jenkins-hbase4:38799] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38799,1692291381679/jenkins-hbase4.apache.org%2C38799%2C1692291381679.jenkins-hbase4.apache.org%2C38799%2C1692291381679.regiongroup-0.1692291382319 2023-08-17 16:56:22,389 INFO [RS:1;jenkins-hbase4:38843] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706/jenkins-hbase4.apache.org%2C38843%2C1692291381706.jenkins-hbase4.apache.org%2C38843%2C1692291381706.regiongroup-0.1692291382319 2023-08-17 16:56:22,390 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36658, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-17 16:56:22,390 DEBUG [RS:0;jenkins-hbase4:38799] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK], DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK], DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK]] 2023-08-17 16:56:22,393 DEBUG [RS:1;jenkins-hbase4:38843] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK], DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK]] 2023-08-17 16:56:22,401 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-17 16:56:22,401 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:22,401 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:22,401 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:22,404 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38843%2C1692291381706.meta, suffix=.meta, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:56:22,420 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:56:22,422 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:56:22,422 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:22,429 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706/jenkins-hbase4.apache.org%2C38843%2C1692291381706.meta.1692291382405.meta 2023-08-17 16:56:22,430 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK], DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK]] 2023-08-17 16:56:22,430 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:22,430 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-17 16:56:22,430 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-17 16:56:22,438 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-17 16:56:22,438 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-17 16:56:22,438 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:22,438 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-17 16:56:22,438 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-17 16:56:22,441 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-17 16:56:22,442 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info 2023-08-17 16:56:22,442 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info 2023-08-17 16:56:22,443 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-17 16:56:22,444 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:22,444 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-17 16:56:22,445 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:56:22,445 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:56:22,445 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-17 16:56:22,447 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:22,447 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-17 16:56:22,448 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table 2023-08-17 16:56:22,448 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table 2023-08-17 16:56:22,448 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-17 16:56:22,449 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:22,451 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740 2023-08-17 16:56:22,454 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740 2023-08-17 16:56:22,456 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-17 16:56:22,458 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-17 16:56:22,459 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63091190, jitterRate=-0.05986800789833069}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-17 16:56:22,459 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-17 16:56:22,460 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692291382387 2023-08-17 16:56:22,470 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-17 16:56:22,471 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-17 16:56:22,472 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38843,1692291381706, state=OPEN 2023-08-17 16:56:22,477 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-17 16:56:22,478 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-17 16:56:22,481 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-17 16:56:22,482 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38843,1692291381706 in 249 msec 2023-08-17 16:56:22,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-17 16:56:22,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 416 msec 2023-08-17 16:56:22,488 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 505 msec 2023-08-17 16:56:22,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692291382489, completionTime=-1 2023-08-17 16:56:22,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-08-17 16:56:22,489 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-17 16:56:22,493 DEBUG [hconnection-0x57392f81-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:22,495 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36668, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:22,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-08-17 16:56:22,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692291442498 2023-08-17 16:56:22,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692291502498 2023-08-17 16:56:22,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 9 msec 2023-08-17 16:56:22,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34345,1692291381639-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34345,1692291381639-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34345,1692291381639-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:34345, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:22,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-17 16:56:22,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:22,509 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-17 16:56:22,509 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-17 16:56:22,512 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-17 16:56:22,513 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-17 16:56:22,518 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:22,519 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4 empty. 2023-08-17 16:56:22,519 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:22,519 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-17 16:56:22,576 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-17 16:56:22,577 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 37b9df605a0d48d3a0fcac9d8ab4c0c4, NAME => 'hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp 2023-08-17 16:56:22,603 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:22,603 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 37b9df605a0d48d3a0fcac9d8ab4c0c4, disabling compactions & flushes 2023-08-17 16:56:22,603 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:22,603 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:22,603 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. after waiting 0 ms 2023-08-17 16:56:22,603 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:22,603 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:22,603 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 37b9df605a0d48d3a0fcac9d8ab4c0c4: 2023-08-17 16:56:22,608 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-17 16:56:22,610 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291382609"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692291382609"}]},"ts":"1692291382609"} 2023-08-17 16:56:22,613 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-17 16:56:22,615 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-17 16:56:22,615 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291382615"}]},"ts":"1692291382615"} 2023-08-17 16:56:22,617 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-17 16:56:22,622 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-17 16:56:22,623 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-17 16:56:22,623 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-17 16:56:22,623 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-17 16:56:22,623 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-17 16:56:22,623 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-17 16:56:22,623 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN}] 2023-08-17 16:56:22,626 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN 2023-08-17 16:56:22,628 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38799,1692291381679; forceNewPlan=false, retain=false 2023-08-17 16:56:22,778 INFO [jenkins-hbase4:34345] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-17 16:56:22,779 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=37b9df605a0d48d3a0fcac9d8ab4c0c4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,780 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291382779"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291382779"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291382779"}]},"ts":"1692291382779"} 2023-08-17 16:56:22,782 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 37b9df605a0d48d3a0fcac9d8ab4c0c4, server=jenkins-hbase4.apache.org,38799,1692291381679}] 2023-08-17 16:56:22,937 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,937 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:22,939 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36770, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-17 16:56:22,945 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:22,945 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:22,947 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38799%2C1692291381679.jenkins-hbase4.apache.org%2C38799%2C1692291381679.regiongroup-1, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38799,1692291381679, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:56:22,963 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:56:22,964 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:22,964 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK] 2023-08-17 16:56:22,968 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38799,1692291381679/jenkins-hbase4.apache.org%2C38799%2C1692291381679.jenkins-hbase4.apache.org%2C38799%2C1692291381679.regiongroup-1.1692291382948 2023-08-17 16:56:22,968 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK], DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK]] 2023-08-17 16:56:22,968 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 37b9df605a0d48d3a0fcac9d8ab4c0c4, NAME => 'hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:22,969 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:22,969 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:22,969 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:22,969 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:22,971 INFO [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:22,973 DEBUG [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/info 2023-08-17 16:56:22,973 DEBUG [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/info 2023-08-17 16:56:22,975 INFO [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 37b9df605a0d48d3a0fcac9d8ab4c0c4 columnFamilyName info 2023-08-17 16:56:22,976 INFO [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] regionserver.HStore(310): Store=37b9df605a0d48d3a0fcac9d8ab4c0c4/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:22,977 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:22,978 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:22,982 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:22,984 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-17 16:56:22,985 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 37b9df605a0d48d3a0fcac9d8ab4c0c4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65995261, jitterRate=-0.01659397780895233}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-17 16:56:22,985 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 37b9df605a0d48d3a0fcac9d8ab4c0c4: 2023-08-17 16:56:22,986 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4., pid=6, masterSystemTime=1692291382937 2023-08-17 16:56:22,993 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:22,993 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:22,994 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=37b9df605a0d48d3a0fcac9d8ab4c0c4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:22,994 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291382994"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291382994"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291382994"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291382994"}]},"ts":"1692291382994"} 2023-08-17 16:56:22,999 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-17 16:56:22,999 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 37b9df605a0d48d3a0fcac9d8ab4c0c4, server=jenkins-hbase4.apache.org,38799,1692291381679 in 215 msec 2023-08-17 16:56:23,003 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-17 16:56:23,004 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN in 376 msec 2023-08-17 16:56:23,005 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-17 16:56:23,005 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291383005"}]},"ts":"1692291383005"} 2023-08-17 16:56:23,007 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-17 16:56:23,010 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-17 16:56:23,010 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-08-17 16:56:23,014 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 503 msec 2023-08-17 16:56:23,014 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-08-17 16:56:23,014 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:23,017 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:23,019 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36776, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:23,021 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-17 16:56:23,033 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-08-17 16:56:23,038 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 16 msec 2023-08-17 16:56:23,044 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-17 16:56:23,053 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-08-17 16:56:23,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 13 msec 2023-08-17 16:56:23,068 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-08-17 16:56:23,071 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-08-17 16:56:23,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.285sec 2023-08-17 16:56:23,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-17 16:56:23,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-17 16:56:23,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-17 16:56:23,073 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34345,1692291381639-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-17 16:56:23,073 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34345,1692291381639-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-17 16:56:23,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-17 16:56:23,077 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(139): Connect 0x037ad110 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:23,082 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@48989e1b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:23,084 DEBUG [hconnection-0x55e24911-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:23,086 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36670, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:23,087 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,34345,1692291381639 2023-08-17 16:56:23,097 DEBUG [Listener at localhost/35639] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-17 16:56:23,098 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50816, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-17 16:56:23,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:23,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-08-17 16:56:23,114 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-17 16:56:23,116 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-17 16:56:23,118 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:23,119 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72 empty. 2023-08-17 16:56:23,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-08-17 16:56:23,119 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:23,119 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-08-17 16:56:23,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-17 16:56:23,143 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-08-17 16:56:23,144 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4b04d98f902897a06d2d5c032dc77d72, NAME => 't1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp 2023-08-17 16:56:23,157 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:23,157 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 4b04d98f902897a06d2d5c032dc77d72, disabling compactions & flushes 2023-08-17 16:56:23,157 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:23,157 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:23,157 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. after waiting 0 ms 2023-08-17 16:56:23,157 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:23,157 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:23,157 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 4b04d98f902897a06d2d5c032dc77d72: 2023-08-17 16:56:23,161 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-17 16:56:23,163 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291383162"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692291383162"}]},"ts":"1692291383162"} 2023-08-17 16:56:23,165 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-17 16:56:23,166 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-17 16:56:23,166 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291383166"}]},"ts":"1692291383166"} 2023-08-17 16:56:23,168 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-08-17 16:56:23,172 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-17 16:56:23,174 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-17 16:56:23,174 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-17 16:56:23,174 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-17 16:56:23,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=4b04d98f902897a06d2d5c032dc77d72, ASSIGN}] 2023-08-17 16:56:23,176 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=4b04d98f902897a06d2d5c032dc77d72, ASSIGN 2023-08-17 16:56:23,177 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=4b04d98f902897a06d2d5c032dc77d72, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42469,1692291378095; forceNewPlan=false, retain=false 2023-08-17 16:56:23,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-17 16:56:23,327 INFO [jenkins-hbase4:33151] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-17 16:56:23,329 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=4b04d98f902897a06d2d5c032dc77d72, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:23,329 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291383329"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291383329"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291383329"}]},"ts":"1692291383329"} 2023-08-17 16:56:23,331 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 4b04d98f902897a06d2d5c032dc77d72, server=jenkins-hbase4.apache.org,42469,1692291378095}] 2023-08-17 16:56:23,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-17 16:56:23,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:23,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4b04d98f902897a06d2d5c032dc77d72, NAME => 't1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:23,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:23,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:23,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:23,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:23,493 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:23,495 DEBUG [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/cf1 2023-08-17 16:56:23,495 DEBUG [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/cf1 2023-08-17 16:56:23,495 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b04d98f902897a06d2d5c032dc77d72 columnFamilyName cf1 2023-08-17 16:56:23,496 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] regionserver.HStore(310): Store=4b04d98f902897a06d2d5c032dc77d72/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:23,496 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:23,498 DEBUG [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/norep 2023-08-17 16:56:23,498 DEBUG [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/norep 2023-08-17 16:56:23,498 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b04d98f902897a06d2d5c032dc77d72 columnFamilyName norep 2023-08-17 16:56:23,499 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] regionserver.HStore(310): Store=4b04d98f902897a06d2d5c032dc77d72/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:23,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:23,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:23,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-17 16:56:23,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:23,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-17 16:56:23,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4b04d98f902897a06d2d5c032dc77d72; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61713452, jitterRate=-0.08039790391921997}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-17 16:56:23,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4b04d98f902897a06d2d5c032dc77d72: 2023-08-17 16:56:23,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72., pid=11, masterSystemTime=1692291383484 2023-08-17 16:56:23,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:23,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:23,513 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=4b04d98f902897a06d2d5c032dc77d72, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:23,513 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291383513"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291383513"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291383513"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291383513"}]},"ts":"1692291383513"} 2023-08-17 16:56:23,518 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-17 16:56:23,518 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 4b04d98f902897a06d2d5c032dc77d72, server=jenkins-hbase4.apache.org,42469,1692291378095 in 185 msec 2023-08-17 16:56:23,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-17 16:56:23,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=4b04d98f902897a06d2d5c032dc77d72, ASSIGN in 344 msec 2023-08-17 16:56:23,523 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-17 16:56:23,523 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291383523"}]},"ts":"1692291383523"} 2023-08-17 16:56:23,524 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-08-17 16:56:23,527 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-17 16:56:23,529 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 417 msec 2023-08-17 16:56:23,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-17 16:56:23,741 INFO [Listener at localhost/35639] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-08-17 16:56:23,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:23,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-08-17 16:56:23,747 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-17 16:56:23,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-08-17 16:56:23,749 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-17 16:56:23,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-17 16:56:23,751 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:23,752 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f empty. 2023-08-17 16:56:23,752 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:23,752 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-08-17 16:56:23,768 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-08-17 16:56:23,769 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => a2668222e397f37ad3389946a7fe032f, NAME => 't2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.tmp 2023-08-17 16:56:23,785 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:23,785 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing a2668222e397f37ad3389946a7fe032f, disabling compactions & flushes 2023-08-17 16:56:23,785 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:23,785 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:23,785 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. after waiting 0 ms 2023-08-17 16:56:23,785 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:23,785 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:23,785 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for a2668222e397f37ad3389946a7fe032f: 2023-08-17 16:56:23,789 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-17 16:56:23,790 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291383790"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692291383790"}]},"ts":"1692291383790"} 2023-08-17 16:56:23,792 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-17 16:56:23,793 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-17 16:56:23,794 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291383793"}]},"ts":"1692291383793"} 2023-08-17 16:56:23,795 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-08-17 16:56:23,799 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-17 16:56:23,800 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-17 16:56:23,800 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-17 16:56:23,800 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-17 16:56:23,800 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=a2668222e397f37ad3389946a7fe032f, ASSIGN}] 2023-08-17 16:56:23,802 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=a2668222e397f37ad3389946a7fe032f, ASSIGN 2023-08-17 16:56:23,803 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=a2668222e397f37ad3389946a7fe032f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45379,1692291378053; forceNewPlan=false, retain=false 2023-08-17 16:56:23,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-17 16:56:23,953 INFO [jenkins-hbase4:33151] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-17 16:56:23,954 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a2668222e397f37ad3389946a7fe032f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:23,954 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291383954"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291383954"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291383954"}]},"ts":"1692291383954"} 2023-08-17 16:56:23,957 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure a2668222e397f37ad3389946a7fe032f, server=jenkins-hbase4.apache.org,45379,1692291378053}] 2023-08-17 16:56:24,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-17 16:56:24,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:24,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:24,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1, suffix=, logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053, archiveDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs, maxLogs=10 2023-08-17 16:56:24,132 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK] 2023-08-17 16:56:24,132 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK] 2023-08-17 16:56:24,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118 2023-08-17 16:56:24,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK], DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK]] 2023-08-17 16:56:24,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a2668222e397f37ad3389946a7fe032f, NAME => 't2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:24,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:24,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:24,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:24,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:24,138 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:24,140 DEBUG [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/cf1 2023-08-17 16:56:24,140 DEBUG [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/cf1 2023-08-17 16:56:24,140 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2668222e397f37ad3389946a7fe032f columnFamilyName cf1 2023-08-17 16:56:24,141 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] regionserver.HStore(310): Store=a2668222e397f37ad3389946a7fe032f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:24,141 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:24,143 DEBUG [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/norep 2023-08-17 16:56:24,143 DEBUG [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/norep 2023-08-17 16:56:24,143 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2668222e397f37ad3389946a7fe032f columnFamilyName norep 2023-08-17 16:56:24,144 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] regionserver.HStore(310): Store=a2668222e397f37ad3389946a7fe032f/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:24,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:24,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:24,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-17 16:56:24,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:24,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-17 16:56:24,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a2668222e397f37ad3389946a7fe032f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70137550, jitterRate=0.04513093829154968}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-17 16:56:24,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a2668222e397f37ad3389946a7fe032f: 2023-08-17 16:56:24,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f., pid=14, masterSystemTime=1692291384109 2023-08-17 16:56:24,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:24,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:24,158 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a2668222e397f37ad3389946a7fe032f, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:24,159 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291384158"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291384158"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291384158"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291384158"}]},"ts":"1692291384158"} 2023-08-17 16:56:24,163 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-17 16:56:24,163 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure a2668222e397f37ad3389946a7fe032f, server=jenkins-hbase4.apache.org,45379,1692291378053 in 204 msec 2023-08-17 16:56:24,168 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-17 16:56:24,169 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=a2668222e397f37ad3389946a7fe032f, ASSIGN in 363 msec 2023-08-17 16:56:24,170 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-17 16:56:24,170 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291384170"}]},"ts":"1692291384170"} 2023-08-17 16:56:24,172 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-08-17 16:56:24,175 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-17 16:56:24,177 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 432 msec 2023-08-17 16:56:24,303 INFO [jenkins-hbase4:42469Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:24,303 INFO [jenkins-hbase4:45379Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:24,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-17 16:56:24,354 INFO [Listener at localhost/35639] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-08-17 16:56:24,358 DEBUG [Listener at localhost/35639] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-17 16:56:24,360 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49090, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-17 16:56:24,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:24,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-08-17 16:56:24,366 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-17 16:56:24,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-08-17 16:56:24,370 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-17 16:56:24,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-17 16:56:24,372 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:24,373 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1 empty. 2023-08-17 16:56:24,373 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:24,373 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-08-17 16:56:24,404 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-08-17 16:56:24,405 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7b7c5dd31d075846c031f622b5c28bc1, NAME => 't1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp 2023-08-17 16:56:24,418 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:24,418 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 7b7c5dd31d075846c031f622b5c28bc1, disabling compactions & flushes 2023-08-17 16:56:24,418 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:24,419 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:24,419 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. after waiting 0 ms 2023-08-17 16:56:24,419 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:24,419 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:24,419 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 7b7c5dd31d075846c031f622b5c28bc1: 2023-08-17 16:56:24,424 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-17 16:56:24,425 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291384425"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692291384425"}]},"ts":"1692291384425"} 2023-08-17 16:56:24,427 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-17 16:56:24,428 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-17 16:56:24,429 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291384428"}]},"ts":"1692291384428"} 2023-08-17 16:56:24,430 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-08-17 16:56:24,434 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-17 16:56:24,436 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-17 16:56:24,436 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-17 16:56:24,436 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-17 16:56:24,436 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-17 16:56:24,436 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-17 16:56:24,436 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN}] 2023-08-17 16:56:24,438 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN 2023-08-17 16:56:24,439 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38843,1692291381706; forceNewPlan=false, retain=false 2023-08-17 16:56:24,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-17 16:56:24,592 INFO [jenkins-hbase4:34345] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-17 16:56:24,592 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=7b7c5dd31d075846c031f622b5c28bc1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:24,593 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291384592"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291384592"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291384592"}]},"ts":"1692291384592"} 2023-08-17 16:56:24,595 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 7b7c5dd31d075846c031f622b5c28bc1, server=jenkins-hbase4.apache.org,38843,1692291381706}] 2023-08-17 16:56:24,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-17 16:56:24,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:24,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:24,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38843%2C1692291381706.jenkins-hbase4.apache.org%2C38843%2C1692291381706.regiongroup-1, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:56:24,771 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:56:24,771 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:56:24,771 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:24,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706/jenkins-hbase4.apache.org%2C38843%2C1692291381706.jenkins-hbase4.apache.org%2C38843%2C1692291381706.regiongroup-1.1692291384754 2023-08-17 16:56:24,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK], DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK]] 2023-08-17 16:56:24,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7b7c5dd31d075846c031f622b5c28bc1, NAME => 't1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:24,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:24,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:24,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:24,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:24,776 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:24,778 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1 2023-08-17 16:56:24,778 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1 2023-08-17 16:56:24,778 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b7c5dd31d075846c031f622b5c28bc1 columnFamilyName cf1 2023-08-17 16:56:24,779 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(310): Store=7b7c5dd31d075846c031f622b5c28bc1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:24,779 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:24,781 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/norep 2023-08-17 16:56:24,781 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/norep 2023-08-17 16:56:24,781 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b7c5dd31d075846c031f622b5c28bc1 columnFamilyName norep 2023-08-17 16:56:24,782 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(310): Store=7b7c5dd31d075846c031f622b5c28bc1/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:24,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:24,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:24,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-17 16:56:24,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:24,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-17 16:56:24,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7b7c5dd31d075846c031f622b5c28bc1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59804258, jitterRate=-0.10884711146354675}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-17 16:56:24,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7b7c5dd31d075846c031f622b5c28bc1: 2023-08-17 16:56:24,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1., pid=11, masterSystemTime=1692291384747 2023-08-17 16:56:24,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:24,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:24,794 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=7b7c5dd31d075846c031f622b5c28bc1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:24,794 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291384794"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291384794"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291384794"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291384794"}]},"ts":"1692291384794"} 2023-08-17 16:56:24,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-17 16:56:24,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 7b7c5dd31d075846c031f622b5c28bc1, server=jenkins-hbase4.apache.org,38843,1692291381706 in 201 msec 2023-08-17 16:56:24,801 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-17 16:56:24,802 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN in 363 msec 2023-08-17 16:56:24,802 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-17 16:56:24,802 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291384802"}]},"ts":"1692291384802"} 2023-08-17 16:56:24,804 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-08-17 16:56:24,807 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-17 16:56:24,810 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 445 msec 2023-08-17 16:56:24,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-17 16:56:24,975 INFO [Listener at localhost/35639] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-08-17 16:56:24,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:24,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-08-17 16:56:24,980 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-17 16:56:24,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-08-17 16:56:24,981 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-17 16:56:24,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-17 16:56:24,983 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:24,984 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7 empty. 2023-08-17 16:56:24,985 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:24,985 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-08-17 16:56:25,008 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-08-17 16:56:25,009 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => ec395d682a88bce1d4eae00d5f8e72c7, NAME => 't2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.tmp 2023-08-17 16:56:25,026 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:25,026 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing ec395d682a88bce1d4eae00d5f8e72c7, disabling compactions & flushes 2023-08-17 16:56:25,026 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:25,026 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:25,026 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. after waiting 0 ms 2023-08-17 16:56:25,027 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:25,027 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:25,027 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for ec395d682a88bce1d4eae00d5f8e72c7: 2023-08-17 16:56:25,030 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-17 16:56:25,031 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291385031"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692291385031"}]},"ts":"1692291385031"} 2023-08-17 16:56:25,034 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-17 16:56:25,035 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-17 16:56:25,035 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291385035"}]},"ts":"1692291385035"} 2023-08-17 16:56:25,037 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-08-17 16:56:25,043 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-17 16:56:25,045 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-17 16:56:25,045 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-17 16:56:25,045 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-17 16:56:25,045 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-17 16:56:25,045 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-17 16:56:25,046 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN}] 2023-08-17 16:56:25,048 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN 2023-08-17 16:56:25,050 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38799,1692291381679; forceNewPlan=false, retain=false 2023-08-17 16:56:25,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-17 16:56:25,200 INFO [jenkins-hbase4:34345] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-17 16:56:25,201 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=ec395d682a88bce1d4eae00d5f8e72c7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:25,202 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291385201"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291385201"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291385201"}]},"ts":"1692291385201"} 2023-08-17 16:56:25,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure ec395d682a88bce1d4eae00d5f8e72c7, server=jenkins-hbase4.apache.org,38799,1692291381679}] 2023-08-17 16:56:25,209 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-17 16:56:25,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-17 16:56:25,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:25,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ec395d682a88bce1d4eae00d5f8e72c7, NAME => 't2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:25,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:25,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:25,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:25,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:25,368 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:25,373 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1 2023-08-17 16:56:25,373 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1 2023-08-17 16:56:25,373 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec395d682a88bce1d4eae00d5f8e72c7 columnFamilyName cf1 2023-08-17 16:56:25,374 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(310): Store=ec395d682a88bce1d4eae00d5f8e72c7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:25,374 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:25,377 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/norep 2023-08-17 16:56:25,377 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/norep 2023-08-17 16:56:25,378 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec395d682a88bce1d4eae00d5f8e72c7 columnFamilyName norep 2023-08-17 16:56:25,378 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(310): Store=ec395d682a88bce1d4eae00d5f8e72c7/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:25,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:25,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:25,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-17 16:56:25,387 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-17 16:56:25,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:25,389 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-08-17 16:56:25,390 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-08-17 16:56:25,391 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-17 16:56:25,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-17 16:56:25,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ec395d682a88bce1d4eae00d5f8e72c7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66783876, jitterRate=-0.004842698574066162}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-17 16:56:25,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ec395d682a88bce1d4eae00d5f8e72c7: 2023-08-17 16:56:25,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7., pid=14, masterSystemTime=1692291385357 2023-08-17 16:56:25,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:25,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:25,397 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=ec395d682a88bce1d4eae00d5f8e72c7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:25,397 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291385397"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291385397"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291385397"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291385397"}]},"ts":"1692291385397"} 2023-08-17 16:56:25,402 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-17 16:56:25,402 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure ec395d682a88bce1d4eae00d5f8e72c7, server=jenkins-hbase4.apache.org,38799,1692291381679 in 195 msec 2023-08-17 16:56:25,404 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-17 16:56:25,404 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN in 356 msec 2023-08-17 16:56:25,405 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-17 16:56:25,405 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692291385405"}]},"ts":"1692291385405"} 2023-08-17 16:56:25,407 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-08-17 16:56:25,409 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-17 16:56:25,411 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 432 msec 2023-08-17 16:56:25,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34345] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-17 16:56:25,589 INFO [Listener at localhost/35639] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-08-17 16:56:25,590 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(139): Connect 0x6774d1fc to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:25,596 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@56dba6b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:25,596 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(139): Connect 0x7974c63d to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:25,609 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@384e13de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:25,628 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.HMaster(3464): Client=jenkins//172.31.14.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:56119:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-08-17 16:56:25,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-08-17 16:56:25,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-17 16:56:25,662 INFO [PEWorker-2] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-17 16:56:25,663 INFO [PEWorker-2] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@32cc0bf8check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:25,668 DEBUG [PEWorker-2-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@32cc0bf8check-peer-cluster-id0x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:25,669 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@32cc0bf8check-peer-cluster-id-0x1001b2c4f410014 connected 2023-08-17 16:56:25,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-17 16:56:25,772 DEBUG [PEWorker-2-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@32cc0bf8check-peer-cluster-id-0x1001b2c4f410014, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:25,780 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:25,792 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-08-17 16:56:25,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-17 16:56:25,957 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45379] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-08-17 16:56:25,957 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42469] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-08-17 16:56:25,958 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-08-17 16:56:25,958 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-08-17 16:56:25,970 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-17 16:56:25,970 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-17 16:56:25,990 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-17 16:56:25,990 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-17 16:56:26,018 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-08-17 16:56:26,018 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-08-17 16:56:26,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] master.HMaster(3641): Remote procedure done, pid=16 2023-08-17 16:56:26,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33151] master.HMaster(3641): Remote procedure done, pid=17 2023-08-17 16:56:26,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33151] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,42469,1692291378095 suceeded 2023-08-17 16:56:26,021 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33151] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,45379,1692291378053 suceeded 2023-08-17 16:56:26,030 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 233 msec 2023-08-17 16:56:26,031 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-08-17 16:56:26,031 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 233 msec 2023-08-17 16:56:26,031 INFO [PEWorker-3] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:56119:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-08-17 16:56:26,033 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 401 msec 2023-08-17 16:56:26,045 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x7c6c2cd4 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:26,047 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x6c9d61ff to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:26,053 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@238d9022, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:26,053 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@a786e71, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:26,055 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:26,055 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:26,060 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:26,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1001b2c4f410017 connected 2023-08-17 16:56:26,061 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:26,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1001b2c4f410018 connected 2023-08-17 16:56:26,063 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=1067016f-f973-4161-8f76-366ce33cec43 to cluster=54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:26,063 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=1067016f-f973-4161-8f76-366ce33cec43 to cluster=54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:26,065 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1 2023-08-17 16:56:26,065 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0 2023-08-17 16:56:26,073 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:56:26,073 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:56:26,074 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0 2023-08-17 16:56:26,074 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1 2023-08-17 16:56:26,074 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1 2023-08-17 16:56:26,074 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:56:26,074 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:56:26,074 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0 2023-08-17 16:56:26,077 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1 2023-08-17 16:56:26,077 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0 2023-08-17 16:56:26,153 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:26,158 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-17 16:56:26,158 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-17 16:56:26,159 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:56:26,159 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-17 16:56:26,159 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:56:26,159 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:56:26,159 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:56:26,159 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:26,159 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-17 16:56:26,159 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:56:26,159 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:26,159 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1.1692291380362, lastWalPosition=600, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:56:26,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33151] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-17 16:56:26,250 DEBUG [Listener at localhost/35639] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-08-17 16:56:26,252 DEBUG [hconnection-0x546edf4d-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:26,254 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40958, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:26,260 DEBUG [Listener at localhost/35639] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:26,266 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58358, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:26,270 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:26,273 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-17 16:56:26,278 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-17 16:56:26,278 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:26,384 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:26,385 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-17 16:56:26,394 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:26,412 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 8764 2023-08-17 16:56:26,412 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 80 WAL entries eligible for replication 2023-08-17 16:56:26,412 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 8764 2023-08-17 16:56:26,413 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4b04d98f902897a06d2d5c032dc77d72/4=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/5=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/6=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/7=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/8=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/9=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/10=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/11=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/12=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/13=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/14=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/15=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/16=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/17=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/18=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/19=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/20=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/21=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/22=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/23=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/24=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/25=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/26=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/27=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/28=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/29=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/30=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/31=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/32=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/33=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/34=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/35=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/36=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/37=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/38=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/39=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/40=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/41=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/42=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/43=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/44=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/45=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/46=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/47=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/48=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/49=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/50=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/51=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/52=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/53=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/54=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/55=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/56=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/57=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/58=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/59=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/60=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/61=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/62=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/63=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/64=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/65=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/66=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/67=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/68=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/69=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/70=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/71=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/72=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/73=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/74=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/75=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/76=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/77=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/78=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/79=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/80=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/81=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/82=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/83=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424, lastWalPosition=8764, nbRowKeys=80, nbHFiles=0, heapSize=16640, lastSeqIds={}, endOfFile=false,usedBufferSize=16640] 2023-08-17 16:56:26,432 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:26,438 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55076, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-08-17 16:56:26,444 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38799] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-17 16:56:26,445 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38799] zookeeper.ReadOnlyZKClient(139): Connect 0x2c2919ad to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:26,452 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38799] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1068c9a7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:26,463 DEBUG [hconnection-0x7dd1fa13-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:26,464 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58372, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-08-17 16:56:26,495 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:26,508 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:26,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 4039 2023-08-17 16:56:26,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 35 WAL entries eligible for replication 2023-08-17 16:56:26,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 4039 2023-08-17 16:56:26,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/a2668222e397f37ad3389946a7fe032f/4=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/5=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/6=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/7=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/8=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/9=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/10=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/11=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/12=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/13=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/14=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/15=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/16=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/17=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/18=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/19=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/20=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/21=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/22=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/23=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/24=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/25=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/26=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/27=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/28=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/29=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/30=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/31=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/32=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/33=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/34=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/35=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/36=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/37=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/38=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118, lastWalPosition=4039, nbRowKeys=35, nbHFiles=0, heapSize=7280, lastSeqIds={}, endOfFile=false,usedBufferSize=7280] 2023-08-17 16:56:26,516 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:26,524 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54574, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-08-17 16:56:26,526 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33525] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-17 16:56:26,527 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33525] zookeeper.ReadOnlyZKClient(139): Connect 0x3874c8b4 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:26,533 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:26,533 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 20 WAL entries eligible for replication 2023-08-17 16:56:26,533 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:26,539 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33525] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3e6c03e6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:26,545 DEBUG [hconnection-0x906f31e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:26,546 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58388, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-08-17 16:56:26,548 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38799] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-17 16:56:26,552 DEBUG [hconnection-0x906f31e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:26,554 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55090, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-08-17 16:56:26,557 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4b04d98f902897a06d2d5c032dc77d72/84=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/85=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/86=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/87=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/88=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/89=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/90=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/91=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/92=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/93=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/94=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/95=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/96=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/97=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/98=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/99=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/100=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/101=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/102=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424, lastWalPosition=10975, nbRowKeys=20, nbHFiles=0, heapSize=4160, lastSeqIds={}, endOfFile=false,usedBufferSize=4160] 2023-08-17 16:56:26,561 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:26,563 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33525] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-17 16:56:26,564 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54582, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-08-17 16:56:26,565 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33525] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-17 16:56:26,572 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33525] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-17 16:56:26,609 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:26,618 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:26,633 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10444 2023-08-17 16:56:26,634 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 61 WAL entries eligible for replication 2023-08-17 16:56:26,634 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10444 2023-08-17 16:56:26,634 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/a2668222e397f37ad3389946a7fe032f/39=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/40=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/41=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/42=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/43=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/44=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/45=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/46=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/47=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/48=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/49=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/50=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/51=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/52=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/53=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/54=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/55=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/56=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/57=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/58=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/59=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/60=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/61=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/62=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/63=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/64=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/65=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/66=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/67=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/68=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/69=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/70=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/71=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/72=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/73=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/74=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/75=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/76=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/77=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/78=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/79=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/80=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/81=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/82=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/83=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/84=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/85=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/86=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/87=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/88=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/89=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/90=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/91=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/92=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/93=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/94=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/95=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/96=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/97=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/98=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/99=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118, lastWalPosition=10444, nbRowKeys=61, nbHFiles=0, heapSize=12688, lastSeqIds={}, endOfFile=false,usedBufferSize=12688] 2023-08-17 16:56:26,644 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33525] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-17 16:56:26,651 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:26,656 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33525] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-17 16:56:26,718 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:26,725 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:26,760 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:26,763 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 18724 2023-08-17 16:56:26,764 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 77 WAL entries eligible for replication 2023-08-17 16:56:26,764 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 18724 2023-08-17 16:56:26,764 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/a2668222e397f37ad3389946a7fe032f/100=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/101=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/102=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/103=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/104=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/105=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/106=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/107=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/108=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/109=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/110=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/111=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/112=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/113=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/114=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/115=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/116=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/117=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/118=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/119=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/120=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/121=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/122=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/123=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/124=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/125=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/126=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/127=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/128=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/129=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/130=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/131=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/132=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/133=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/134=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/135=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/136=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/137=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/138=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/139=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/140=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/141=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/142=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/143=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/144=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/145=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/146=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/147=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/148=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/149=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/150=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/151=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/152=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/153=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/154=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/155=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/156=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/157=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/158=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/159=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/160=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/161=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/162=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/163=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/164=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/165=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/166=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/167=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/168=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/169=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/170=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/171=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/172=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/173=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/174=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/175=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/176=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118, lastWalPosition=18724, nbRowKeys=77, nbHFiles=0, heapSize=16016, lastSeqIds={}, endOfFile=false,usedBufferSize=16016] 2023-08-17 16:56:26,766 DEBUG [SinkThread-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:26,768 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55106, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-08-17 16:56:26,770 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38799] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-17 16:56:26,774 DEBUG [hconnection-0x7dd1fa13-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:26,776 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55114, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-08-17 16:56:26,781 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38799] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-17 16:56:26,826 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:26,832 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:26,867 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:26,876 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:26,876 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 27 WAL entries eligible for replication 2023-08-17 16:56:26,876 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:26,876 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/a2668222e397f37ad3389946a7fe032f/177=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/178=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/179=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/180=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/181=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/182=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/183=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/184=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/185=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/186=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/187=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/188=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/189=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/190=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/191=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/192=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/193=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/194=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/195=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/196=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/197=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/198=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/199=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/200=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/201=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/202=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118, lastWalPosition=21752, nbRowKeys=27, nbHFiles=0, heapSize=5616, lastSeqIds={}, endOfFile=false,usedBufferSize=5616] 2023-08-17 16:56:26,877 DEBUG [SinkThread-3] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:26,878 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58392, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-08-17 16:56:26,880 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38843] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-17 16:56:26,880 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38843] zookeeper.ReadOnlyZKClient(139): Connect 0x0253aaed to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:26,886 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38843] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@c9444d7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:26,887 DEBUG [hconnection-0x8abc19d-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:26,889 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58404, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-08-17 16:56:26,894 DEBUG [hconnection-0x8abc19d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:26,895 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55128, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-08-17 16:56:26,898 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38843] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-17 16:56:26,935 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:26,941 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:26,974 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:26,985 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:27,042 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:27,048 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:27,066 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-17 16:56:27,067 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-17 16:56:27,080 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:27,092 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:27,149 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:27,154 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:27,186 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:27,201 INFO [jenkins-hbase4:33525Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:27,202 INFO [jenkins-hbase4:33525Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 116 2023-08-17 16:56:27,202 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:27,207 INFO [jenkins-hbase4:41501Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:27,211 INFO [jenkins-hbase4:38799Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:27,211 INFO [jenkins-hbase4:38799Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 157 2023-08-17 16:56:27,212 INFO [jenkins-hbase4:38843Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:27,212 INFO [jenkins-hbase4:38843Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 27 2023-08-17 16:56:27,255 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:27,259 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:27,291 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:27,297 DEBUG [hconnection-0x5018485e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:27,298 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58418, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:27,308 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:27,317 DEBUG [hconnection-0x5018485e-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:27,319 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55136, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:27,326 DEBUG [Listener at localhost/35639] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-08-17 16:56:27,326 INFO [Listener at localhost/35639] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-17 16:56:27,327 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x037ad110 to 127.0.0.1:56119 2023-08-17 16:56:27,327 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,328 DEBUG [Listener at localhost/35639] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-17 16:56:27,328 DEBUG [Listener at localhost/35639] util.JVMClusterUtil(257): Found active master hash=74855691, stopped=false 2023-08-17 16:56:27,328 INFO [Listener at localhost/35639] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,34345,1692291381639 2023-08-17 16:56:27,331 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-17 16:56:27,331 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-17 16:56:27,331 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-17 16:56:27,331 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:27,331 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-17 16:56:27,331 INFO [Listener at localhost/35639] procedure2.ProcedureExecutor(629): Stopping 2023-08-17 16:56:27,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:27,331 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-17 16:56:27,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:27,332 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2190a892 to 127.0.0.1:56119 2023-08-17 16:56:27,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:27,333 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:27,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:27,333 INFO [Listener at localhost/35639] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38799,1692291381679' ***** 2023-08-17 16:56:27,334 INFO [Listener at localhost/35639] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-17 16:56:27,334 INFO [Listener at localhost/35639] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38843,1692291381706' ***** 2023-08-17 16:56:27,334 INFO [Listener at localhost/35639] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-17 16:56:27,334 INFO [Listener at localhost/35639] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33525,1692291381734' ***** 2023-08-17 16:56:27,334 INFO [Listener at localhost/35639] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-17 16:56:27,334 INFO [Listener at localhost/35639] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41501,1692291381758' ***** 2023-08-17 16:56:27,334 INFO [Listener at localhost/35639] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-17 16:56:27,334 INFO [RS:1;jenkins-hbase4:38843] regionserver.HeapMemoryManager(220): Stopping 2023-08-17 16:56:27,335 INFO [RS:1;jenkins-hbase4:38843] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-17 16:56:27,335 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-17 16:56:27,335 INFO [RS:1;jenkins-hbase4:38843] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-17 16:56:27,334 INFO [RS:3;jenkins-hbase4:41501] regionserver.HeapMemoryManager(220): Stopping 2023-08-17 16:56:27,334 INFO [RS:0;jenkins-hbase4:38799] regionserver.HeapMemoryManager(220): Stopping 2023-08-17 16:56:27,336 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-17 16:56:27,334 INFO [RS:2;jenkins-hbase4:33525] regionserver.HeapMemoryManager(220): Stopping 2023-08-17 16:56:27,336 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-17 16:56:27,336 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-17 16:56:27,336 INFO [RS:2;jenkins-hbase4:33525] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-17 16:56:27,336 INFO [RS:0;jenkins-hbase4:38799] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-17 16:56:27,336 INFO [RS:3;jenkins-hbase4:41501] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-17 16:56:27,336 INFO [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(3305): Received CLOSE for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:27,338 INFO [RS:3;jenkins-hbase4:41501] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-17 16:56:27,338 INFO [RS:0;jenkins-hbase4:38799] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-17 16:56:27,338 INFO [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:27,338 INFO [RS:2;jenkins-hbase4:33525] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-17 16:56:27,339 DEBUG [RS:3;jenkins-hbase4:41501] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2636e685 to 127.0.0.1:56119 2023-08-17 16:56:27,339 INFO [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:27,338 INFO [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(3305): Received CLOSE for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:27,339 DEBUG [RS:2;jenkins-hbase4:33525] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x704bb248 to 127.0.0.1:56119 2023-08-17 16:56:27,339 INFO [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:27,339 DEBUG [RS:3;jenkins-hbase4:41501] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ec395d682a88bce1d4eae00d5f8e72c7, disabling compactions & flushes 2023-08-17 16:56:27,341 INFO [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41501,1692291381758; all regions closed. 2023-08-17 16:56:27,340 DEBUG [RS:1;jenkins-hbase4:38843] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4c597470 to 127.0.0.1:56119 2023-08-17 16:56:27,340 INFO [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(3305): Received CLOSE for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:27,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7b7c5dd31d075846c031f622b5c28bc1, disabling compactions & flushes 2023-08-17 16:56:27,341 INFO [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:27,339 DEBUG [RS:2;jenkins-hbase4:33525] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,341 DEBUG [RS:0;jenkins-hbase4:38799] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6dcdd537 to 127.0.0.1:56119 2023-08-17 16:56:27,341 INFO [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33525,1692291381734; all regions closed. 2023-08-17 16:56:27,341 DEBUG [RS:0;jenkins-hbase4:38799] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,341 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:27,341 DEBUG [RS:1;jenkins-hbase4:38843] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,341 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:27,342 INFO [RS:1;jenkins-hbase4:38843] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-17 16:56:27,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:27,341 INFO [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-08-17 16:56:27,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. after waiting 0 ms 2023-08-17 16:56:27,342 DEBUG [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1478): Online Regions={ec395d682a88bce1d4eae00d5f8e72c7=t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7., 37b9df605a0d48d3a0fcac9d8ab4c0c4=hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.} 2023-08-17 16:56:27,342 INFO [RS:1;jenkins-hbase4:38843] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-17 16:56:27,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:27,342 INFO [RS:1;jenkins-hbase4:38843] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-17 16:56:27,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:27,342 INFO [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-17 16:56:27,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. after waiting 0 ms 2023-08-17 16:56:27,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:27,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 7b7c5dd31d075846c031f622b5c28bc1 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-08-17 16:56:27,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ec395d682a88bce1d4eae00d5f8e72c7 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-08-17 16:56:27,343 DEBUG [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1504): Waiting on 37b9df605a0d48d3a0fcac9d8ab4c0c4, ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:27,348 INFO [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-08-17 16:56:27,348 DEBUG [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1478): Online Regions={7b7c5dd31d075846c031f622b5c28bc1=t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1., 1588230740=hbase:meta,,1.1588230740} 2023-08-17 16:56:27,349 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-17 16:56:27,349 DEBUG [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1504): Waiting on 1588230740, 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:27,349 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-17 16:56:27,349 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-17 16:56:27,349 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-17 16:56:27,349 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-17 16:56:27,349 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-08-17 16:56:27,368 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:27,372 DEBUG [RS:2;jenkins-hbase4:33525] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:56:27,373 INFO [RS:2;jenkins-hbase4:33525] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33525%2C1692291381734.jenkins-hbase4.apache.org%2C33525%2C1692291381734.regiongroup-0:(num 1692291382309) 2023-08-17 16:56:27,373 DEBUG [RS:2;jenkins-hbase4:33525] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,373 INFO [RS:2;jenkins-hbase4:33525] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:27,373 DEBUG [RS:3;jenkins-hbase4:41501] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:56:27,373 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:27,373 INFO [RS:2;jenkins-hbase4:33525] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-17 16:56:27,373 INFO [RS:3;jenkins-hbase4:41501] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41501%2C1692291381758.jenkins-hbase4.apache.org%2C41501%2C1692291381758.regiongroup-0:(num 1692291382312) 2023-08-17 16:56:27,373 INFO [RS:2;jenkins-hbase4:33525] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-17 16:56:27,373 INFO [RS:2;jenkins-hbase4:33525] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-17 16:56:27,374 INFO [RS:2;jenkins-hbase4:33525] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-17 16:56:27,373 DEBUG [RS:3;jenkins-hbase4:41501] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,374 INFO [RS:3;jenkins-hbase4:41501] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:27,374 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:56:27,374 DEBUG [RS:2;jenkins-hbase4:33525] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3874c8b4 to 127.0.0.1:56119 2023-08-17 16:56:27,375 DEBUG [RS:2;jenkins-hbase4:33525] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,377 INFO [RS:2;jenkins-hbase4:33525] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33525 2023-08-17 16:56:27,378 INFO [RS:3;jenkins-hbase4:41501] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-17 16:56:27,380 INFO [RS:3;jenkins-hbase4:41501] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-17 16:56:27,380 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:56:27,380 INFO [RS:3;jenkins-hbase4:41501] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-17 16:56:27,380 INFO [RS:3;jenkins-hbase4:41501] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-17 16:56:27,381 INFO [RS:3;jenkins-hbase4:41501] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41501 2023-08-17 16:56:27,383 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:27,386 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:27,389 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:27,389 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:27,389 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:27,389 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,389 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,389 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410018, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,389 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410017, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,389 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,390 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:56:27,389 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,41501,1692291381758 2023-08-17 16:56:27,390 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,390 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:56:27,389 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,396 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41501,1692291381758] 2023-08-17 16:56:27,396 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41501,1692291381758; numProcessing=1 2023-08-17 16:56:27,398 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410017, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,398 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:27,398 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:56:27,398 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410018, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,398 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:27,398 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:56:27,398 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,399 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:27,398 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33525,1692291381734 2023-08-17 16:56:27,399 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,41501,1692291381758 already deleted, retry=false 2023-08-17 16:56:27,399 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41501,1692291381758 expired; onlineServers=3 2023-08-17 16:56:27,400 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33525,1692291381734] 2023-08-17 16:56:27,400 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33525,1692291381734; numProcessing=2 2023-08-17 16:56:27,401 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,33525,1692291381734 already deleted, retry=false 2023-08-17 16:56:27,401 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33525,1692291381734 expired; onlineServers=2 2023-08-17 16:56:27,402 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:27,403 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:27,413 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:27,446 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/.tmp/info/d862d25bca50496b8832b846a15b761e 2023-08-17 16:56:27,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=8 (bloomFilter=true), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/.tmp/cf1/5a8be9b14e044fb4be92342ffecc0175 2023-08-17 16:56:27,450 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/.tmp/cf1/36b4336e2c014c019e4c7d9f163363d2 2023-08-17 16:56:27,476 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:27,478 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:27,491 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/.tmp/cf1/36b4336e2c014c019e4c7d9f163363d2 as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1/36b4336e2c014c019e4c7d9f163363d2 2023-08-17 16:56:27,491 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/.tmp/cf1/5a8be9b14e044fb4be92342ffecc0175 as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1/5a8be9b14e044fb4be92342ffecc0175 2023-08-17 16:56:27,503 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1/36b4336e2c014c019e4c7d9f163363d2, entries=100, sequenceid=6, filesize=8.3 K 2023-08-17 16:56:27,505 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1/5a8be9b14e044fb4be92342ffecc0175, entries=200, sequenceid=8, filesize=12.2 K 2023-08-17 16:56:27,508 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for ec395d682a88bce1d4eae00d5f8e72c7 in 165ms, sequenceid=8, compaction requested=false 2023-08-17 16:56:27,508 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 7b7c5dd31d075846c031f622b5c28bc1 in 165ms, sequenceid=6, compaction requested=false 2023-08-17 16:56:27,511 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/.tmp/table/0a63d2c9dbf3474c8afb858114c8a01a 2023-08-17 16:56:27,521 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:27,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/recovered.edits/11.seqid, newMaxSeqId=11, maxSeqId=1 2023-08-17 16:56:27,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:27,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ec395d682a88bce1d4eae00d5f8e72c7: 2023-08-17 16:56:27,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:27,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 37b9df605a0d48d3a0fcac9d8ab4c0c4, disabling compactions & flushes 2023-08-17 16:56:27,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:27,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:27,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. after waiting 0 ms 2023-08-17 16:56:27,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:27,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 37b9df605a0d48d3a0fcac9d8ab4c0c4 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-17 16:56:27,531 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:27,532 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:27,532 INFO [RS:2;jenkins-hbase4:33525] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33525,1692291381734; zookeeper connection closed. 2023-08-17 16:56:27,532 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:33525-0x1001b2c4f41000a, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:27,533 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/.tmp/info/d862d25bca50496b8832b846a15b761e as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info/d862d25bca50496b8832b846a15b761e 2023-08-17 16:56:27,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-17 16:56:27,539 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@275b292b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@275b292b 2023-08-17 16:56:27,541 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:27,541 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7b7c5dd31d075846c031f622b5c28bc1: 2023-08-17 16:56:27,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:27,543 DEBUG [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1504): Waiting on 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:27,545 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info/d862d25bca50496b8832b846a15b761e, entries=30, sequenceid=19, filesize=8.1 K 2023-08-17 16:56:27,546 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/.tmp/table/0a63d2c9dbf3474c8afb858114c8a01a as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table/0a63d2c9dbf3474c8afb858114c8a01a 2023-08-17 16:56:27,549 DEBUG [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-08-17 16:56:27,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/.tmp/info/8bfbb4041ee748bc8dba5ffabe3bdbca 2023-08-17 16:56:27,555 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table/0a63d2c9dbf3474c8afb858114c8a01a, entries=6, sequenceid=19, filesize=4.8 K 2023-08-17 16:56:27,556 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 207ms, sequenceid=19, compaction requested=false 2023-08-17 16:56:27,565 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-08-17 16:56:27,565 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-17 16:56:27,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/.tmp/info/8bfbb4041ee748bc8dba5ffabe3bdbca as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/info/8bfbb4041ee748bc8dba5ffabe3bdbca 2023-08-17 16:56:27,566 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-17 16:56:27,566 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-17 16:56:27,566 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-17 16:56:27,571 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/info/8bfbb4041ee748bc8dba5ffabe3bdbca, entries=2, sequenceid=6, filesize=4.8 K 2023-08-17 16:56:27,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 37b9df605a0d48d3a0fcac9d8ab4c0c4 in 44ms, sequenceid=6, compaction requested=false 2023-08-17 16:56:27,580 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-17 16:56:27,581 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:27,581 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 37b9df605a0d48d3a0fcac9d8ab4c0c4: 2023-08-17 16:56:27,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:27,582 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:27,584 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:27,626 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:27,632 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:27,632 INFO [RS:3;jenkins-hbase4:41501] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41501,1692291381758; zookeeper connection closed. 2023-08-17 16:56:27,632 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:41501-0x1001b2c4f41000b, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:27,634 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4c9c34db] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4c9c34db 2023-08-17 16:56:27,638 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:27,687 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:27,688 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:27,731 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:27,742 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:27,744 INFO [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38799,1692291381679; all regions closed. 2023-08-17 16:56:27,750 INFO [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38843,1692291381706; all regions closed. 2023-08-17 16:56:27,753 DEBUG [RS:0;jenkins-hbase4:38799] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:56:27,753 INFO [RS:0;jenkins-hbase4:38799] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38799%2C1692291381679.jenkins-hbase4.apache.org%2C38799%2C1692291381679.regiongroup-0:(num 1692291382319) 2023-08-17 16:56:27,753 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706/jenkins-hbase4.apache.org%2C38843%2C1692291381706.meta.1692291382405.meta not finished, retry = 0 2023-08-17 16:56:27,759 DEBUG [RS:0;jenkins-hbase4:38799] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:56:27,759 INFO [RS:0;jenkins-hbase4:38799] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38799%2C1692291381679.jenkins-hbase4.apache.org%2C38799%2C1692291381679.regiongroup-1:(num 1692291382948) 2023-08-17 16:56:27,759 DEBUG [RS:0;jenkins-hbase4:38799] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,759 INFO [RS:0;jenkins-hbase4:38799] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:27,760 INFO [RS:0;jenkins-hbase4:38799] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-17 16:56:27,760 INFO [RS:0;jenkins-hbase4:38799] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-17 16:56:27,760 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:56:27,760 INFO [RS:0;jenkins-hbase4:38799] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-17 16:56:27,760 INFO [RS:0;jenkins-hbase4:38799] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-17 16:56:27,760 DEBUG [RS:0;jenkins-hbase4:38799] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2c2919ad to 127.0.0.1:56119 2023-08-17 16:56:27,760 DEBUG [RS:0;jenkins-hbase4:38799] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,761 INFO [RS:0;jenkins-hbase4:38799] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38799 2023-08-17 16:56:27,764 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:27,764 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410017, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,764 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,764 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410018, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,764 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,38799,1692291381679 2023-08-17 16:56:27,764 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:56:27,764 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:56:27,765 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38799,1692291381679] 2023-08-17 16:56:27,765 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38799,1692291381679; numProcessing=3 2023-08-17 16:56:27,766 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,38799,1692291381679 already deleted, retry=false 2023-08-17 16:56:27,766 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38799,1692291381679 expired; onlineServers=1 2023-08-17 16:56:27,791 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:27,792 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:27,837 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:27,846 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:27,856 DEBUG [RS:1;jenkins-hbase4:38843] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:56:27,856 INFO [RS:1;jenkins-hbase4:38843] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38843%2C1692291381706.meta:.meta(num 1692291382405) 2023-08-17 16:56:27,865 DEBUG [RS:1;jenkins-hbase4:38843] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:56:27,866 INFO [RS:1;jenkins-hbase4:38843] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38843%2C1692291381706.jenkins-hbase4.apache.org%2C38843%2C1692291381706.regiongroup-1:(num 1692291384754) 2023-08-17 16:56:27,871 DEBUG [RS:1;jenkins-hbase4:38843] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:56:27,871 INFO [RS:1;jenkins-hbase4:38843] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38843%2C1692291381706.jenkins-hbase4.apache.org%2C38843%2C1692291381706.regiongroup-0:(num 1692291382319) 2023-08-17 16:56:27,871 DEBUG [RS:1;jenkins-hbase4:38843] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,871 INFO [RS:1;jenkins-hbase4:38843] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:27,871 INFO [RS:1;jenkins-hbase4:38843] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-17 16:56:27,871 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:56:27,871 DEBUG [RS:1;jenkins-hbase4:38843] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0253aaed to 127.0.0.1:56119 2023-08-17 16:56:27,871 DEBUG [RS:1;jenkins-hbase4:38843] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,872 INFO [RS:1;jenkins-hbase4:38843] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38843 2023-08-17 16:56:27,875 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,38843,1692291381706 2023-08-17 16:56:27,875 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410018, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,875 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,875 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410017, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:27,875 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:56:27,875 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:56:27,876 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38843,1692291381706] 2023-08-17 16:56:27,876 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38843,1692291381706; numProcessing=4 2023-08-17 16:56:27,877 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,38843,1692291381706 already deleted, retry=false 2023-08-17 16:56:27,877 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38843,1692291381706 expired; onlineServers=0 2023-08-17 16:56:27,877 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34345,1692291381639' ***** 2023-08-17 16:56:27,878 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-17 16:56:27,878 DEBUG [M:0;jenkins-hbase4:34345] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49a15d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:56:27,878 INFO [M:0;jenkins-hbase4:34345] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34345,1692291381639 2023-08-17 16:56:27,878 INFO [M:0;jenkins-hbase4:34345] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34345,1692291381639; all regions closed. 2023-08-17 16:56:27,878 DEBUG [M:0;jenkins-hbase4:34345] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:27,878 DEBUG [M:0;jenkins-hbase4:34345] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-17 16:56:27,879 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-17 16:56:27,879 DEBUG [M:0;jenkins-hbase4:34345] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-17 16:56:27,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291381989] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291381989,5,FailOnTimeoutGroup] 2023-08-17 16:56:27,879 INFO [M:0;jenkins-hbase4:34345] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-17 16:56:27,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291381989] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291381989,5,FailOnTimeoutGroup] 2023-08-17 16:56:27,880 INFO [M:0;jenkins-hbase4:34345] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-17 16:56:27,881 INFO [M:0;jenkins-hbase4:34345] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-17 16:56:27,881 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-08-17 16:56:27,881 DEBUG [M:0;jenkins-hbase4:34345] master.HMaster(1512): Stopping service threads 2023-08-17 16:56:27,881 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:27,881 INFO [M:0;jenkins-hbase4:34345] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-17 16:56:27,882 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:56:27,882 ERROR [M:0;jenkins-hbase4:34345] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-08-17 16:56:27,882 INFO [M:0;jenkins-hbase4:34345] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-17 16:56:27,882 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-17 16:56:27,883 DEBUG [M:0;jenkins-hbase4:34345] zookeeper.ZKUtil(398): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-08-17 16:56:27,883 WARN [M:0;jenkins-hbase4:34345] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-17 16:56:27,883 INFO [M:0;jenkins-hbase4:34345] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-17 16:56:27,884 INFO [M:0;jenkins-hbase4:34345] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-17 16:56:27,884 DEBUG [M:0;jenkins-hbase4:34345] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-17 16:56:27,884 INFO [M:0;jenkins-hbase4:34345] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:27,884 DEBUG [M:0;jenkins-hbase4:34345] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:27,884 DEBUG [M:0;jenkins-hbase4:34345] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-17 16:56:27,884 DEBUG [M:0;jenkins-hbase4:34345] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:27,885 INFO [M:0;jenkins-hbase4:34345] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.41 KB heapSize=65.20 KB 2023-08-17 16:56:27,957 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:27,959 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:27,960 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:27,961 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:27,961 INFO [M:0;jenkins-hbase4:34345] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.41 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/0c1abd12533b4b5aae99c934fcfdd5b9 2023-08-17 16:56:27,968 DEBUG [M:0;jenkins-hbase4:34345] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/0c1abd12533b4b5aae99c934fcfdd5b9 as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/0c1abd12533b4b5aae99c934fcfdd5b9 2023-08-17 16:56:27,973 INFO [M:0;jenkins-hbase4:34345] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/0c1abd12533b4b5aae99c934fcfdd5b9, entries=14, sequenceid=116, filesize=8.1 K 2023-08-17 16:56:27,974 INFO [M:0;jenkins-hbase4:34345] regionserver.HRegion(2948): Finished flush of dataSize ~55.41 KB/56744, heapSize ~65.18 KB/66744, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 89ms, sequenceid=116, compaction requested=false 2023-08-17 16:56:27,976 INFO [M:0;jenkins-hbase4:34345] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:27,976 DEBUG [M:0;jenkins-hbase4:34345] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:56:27,979 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:56:27,979 INFO [M:0;jenkins-hbase4:34345] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-17 16:56:27,980 INFO [M:0;jenkins-hbase4:34345] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34345 2023-08-17 16:56:27,982 DEBUG [M:0;jenkins-hbase4:34345] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,34345,1692291381639 already deleted, retry=false 2023-08-17 16:56:28,062 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:28,064 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:28,064 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:28,065 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:28,168 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:28,168 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:28,169 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:28,169 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:28,275 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:28,275 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:28,275 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:28,275 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:28,352 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:28,352 INFO [M:0;jenkins-hbase4:34345] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34345,1692291381639; zookeeper connection closed. 2023-08-17 16:56:28,352 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34345-0x1001b2c4f410007, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:28,380 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:28,380 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:28,381 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:28,381 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:28,452 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:28,452 INFO [RS:1;jenkins-hbase4:38843] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38843,1692291381706; zookeeper connection closed. 2023-08-17 16:56:28,452 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38843-0x1001b2c4f410009, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:28,454 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@476d0c42] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@476d0c42 2023-08-17 16:56:28,485 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:28,485 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:28,485 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:28,486 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:28,590 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-17 16:56:28,590 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-17 16:56:28,590 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:28,591 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-17 16:56:28,653 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:28,653 INFO [RS:0;jenkins-hbase4:38799] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38799,1692291381679; zookeeper connection closed. 2023-08-17 16:56:28,653 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:38799-0x1001b2c4f410008, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:28,654 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@49ce0de1] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@49ce0de1 2023-08-17 16:56:28,654 INFO [Listener at localhost/35639] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-08-17 16:56:28,677 INFO [Listener at localhost/35639] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-17 16:56:28,677 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x185830dc to 127.0.0.1:56119 2023-08-17 16:56:28,677 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:28,677 DEBUG [Listener at localhost/35639] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-17 16:56:28,677 DEBUG [Listener at localhost/35639] util.JVMClusterUtil(257): Found active master hash=2028074190, stopped=false 2023-08-17 16:56:28,677 INFO [Listener at localhost/35639] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33151,1692291376904 2023-08-17 16:56:28,682 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-17 16:56:28,682 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-17 16:56:28,682 INFO [Listener at localhost/35639] procedure2.ProcedureExecutor(629): Stopping 2023-08-17 16:56:28,682 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-17 16:56:28,682 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:28,685 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3dedaa22 to 127.0.0.1:56119 2023-08-17 16:56:28,685 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:28,685 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-17 16:56:28,685 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-17 16:56:28,685 INFO [Listener at localhost/35639] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45379,1692291378053' ***** 2023-08-17 16:56:28,686 INFO [Listener at localhost/35639] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-17 16:56:28,686 INFO [Listener at localhost/35639] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42469,1692291378095' ***** 2023-08-17 16:56:28,686 INFO [Listener at localhost/35639] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-17 16:56:28,686 INFO [RS:1;jenkins-hbase4:42469] regionserver.HeapMemoryManager(220): Stopping 2023-08-17 16:56:28,686 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-17 16:56:28,686 INFO [RS:1;jenkins-hbase4:42469] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-17 16:56:28,686 INFO [RS:1;jenkins-hbase4:42469] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-17 16:56:28,686 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:28,686 INFO [RS:0;jenkins-hbase4:45379] regionserver.HeapMemoryManager(220): Stopping 2023-08-17 16:56:28,687 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-17 16:56:28,687 INFO [RS:0;jenkins-hbase4:45379] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-17 16:56:28,688 INFO [RS:0;jenkins-hbase4:45379] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-17 16:56:28,688 INFO [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(3305): Received CLOSE for a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:28,686 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-17 16:56:28,688 INFO [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:28,686 INFO [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(3305): Received CLOSE for e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:28,689 DEBUG [RS:0;jenkins-hbase4:45379] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x794783db to 127.0.0.1:56119 2023-08-17 16:56:28,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a2668222e397f37ad3389946a7fe032f, disabling compactions & flushes 2023-08-17 16:56:28,688 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:28,690 INFO [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(3305): Received CLOSE for 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:56:28,690 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:28,690 DEBUG [RS:0;jenkins-hbase4:45379] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:28,690 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:28,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. after waiting 0 ms 2023-08-17 16:56:28,690 INFO [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:28,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:28,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e13cf09e05e627f75f15f5ddbd0b03a2, disabling compactions & flushes 2023-08-17 16:56:28,690 INFO [RS:0;jenkins-hbase4:45379] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-17 16:56:28,692 INFO [RS:0;jenkins-hbase4:45379] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-17 16:56:28,692 INFO [RS:0;jenkins-hbase4:45379] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-17 16:56:28,692 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:28,692 INFO [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-17 16:56:28,691 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing a2668222e397f37ad3389946a7fe032f 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-08-17 16:56:28,692 INFO [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-08-17 16:56:28,692 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1478): Online Regions={a2668222e397f37ad3389946a7fe032f=t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f., 1588230740=hbase:meta,,1.1588230740} 2023-08-17 16:56:28,691 DEBUG [RS:1;jenkins-hbase4:42469] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x76311fa0 to 127.0.0.1:56119 2023-08-17 16:56:28,692 DEBUG [RS:1;jenkins-hbase4:42469] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:28,692 INFO [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-08-17 16:56:28,692 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1504): Waiting on 1588230740, a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:28,692 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:28,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. after waiting 0 ms 2023-08-17 16:56:28,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:28,694 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing e13cf09e05e627f75f15f5ddbd0b03a2 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-17 16:56:28,694 DEBUG [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1478): Online Regions={e13cf09e05e627f75f15f5ddbd0b03a2=hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2., 4b04d98f902897a06d2d5c032dc77d72=t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.} 2023-08-17 16:56:28,693 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-17 16:56:28,695 DEBUG [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1504): Waiting on 4b04d98f902897a06d2d5c032dc77d72, e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:56:28,695 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-17 16:56:28,695 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-17 16:56:28,695 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-17 16:56:28,695 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-17 16:56:28,695 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-08-17 16:56:28,698 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 845 2023-08-17 16:56:28,698 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:28,698 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:56:28,698 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 845 2023-08-17 16:56:28,698 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1.1692291380362, lastWalPosition=845, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:56:28,699 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-17 16:56:28,700 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-17 16:56:28,700 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-17 16:56:28,700 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/a2668222e397f37ad3389946a7fe032f/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118, lastWalPosition=28643, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-08-17 16:56:28,709 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-08-17 16:56:28,709 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-17 16:56:28,709 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-08-17 16:56:28,709 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4b04d98f902897a06d2d5c032dc77d72/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424, lastWalPosition=14321, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-08-17 16:56:28,733 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/.tmp/info/0eb76268e85e45b6914ae6bee766a07e 2023-08-17 16:56:28,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/.tmp/info/0eb76268e85e45b6914ae6bee766a07e as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/info/0eb76268e85e45b6914ae6bee766a07e 2023-08-17 16:56:28,751 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/info/0eb76268e85e45b6914ae6bee766a07e, entries=2, sequenceid=6, filesize=4.8 K 2023-08-17 16:56:28,752 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for e13cf09e05e627f75f15f5ddbd0b03a2 in 58ms, sequenceid=6, compaction requested=false 2023-08-17 16:56:28,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-17 16:56:28,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:28,760 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e13cf09e05e627f75f15f5ddbd0b03a2: 2023-08-17 16:56:28,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:56:28,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4b04d98f902897a06d2d5c032dc77d72, disabling compactions & flushes 2023-08-17 16:56:28,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:28,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:28,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. after waiting 0 ms 2023-08-17 16:56:28,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:28,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 4b04d98f902897a06d2d5c032dc77d72 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-08-17 16:56:28,776 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/.tmp/cf1/c1f1861a6a844c60805674304ef77081 2023-08-17 16:56:28,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1f1861a6a844c60805674304ef77081 2023-08-17 16:56:28,796 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/.tmp/norep/17767cd3834547738b3f53cdd6bcb6bf 2023-08-17 16:56:28,803 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17767cd3834547738b3f53cdd6bcb6bf 2023-08-17 16:56:28,804 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-08-17 16:56:28,804 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:56:28,804 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-08-17 16:56:28,806 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-17 16:56:28,810 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/.tmp/cf1/c1f1861a6a844c60805674304ef77081 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/cf1/c1f1861a6a844c60805674304ef77081 2023-08-17 16:56:28,810 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:28,814 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-08-17 16:56:28,814 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:56:28,814 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-08-17 16:56:28,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1f1861a6a844c60805674304ef77081 2023-08-17 16:56:28,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/cf1/c1f1861a6a844c60805674304ef77081, entries=100, sequenceid=106, filesize=8.2 K 2023-08-17 16:56:28,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/.tmp/norep/17767cd3834547738b3f53cdd6bcb6bf as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/norep/17767cd3834547738b3f53cdd6bcb6bf 2023-08-17 16:56:28,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17767cd3834547738b3f53cdd6bcb6bf 2023-08-17 16:56:28,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/norep/17767cd3834547738b3f53cdd6bcb6bf, entries=51, sequenceid=106, filesize=6.5 K 2023-08-17 16:56:28,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for 4b04d98f902897a06d2d5c032dc77d72 in 63ms, sequenceid=106, compaction requested=false 2023-08-17 16:56:28,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-08-17 16:56:28,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:28,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4b04d98f902897a06d2d5c032dc77d72: 2023-08-17 16:56:28,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:56:28,894 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1504): Waiting on 1588230740, a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:28,895 INFO [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42469,1692291378095; all regions closed. 2023-08-17 16:56:28,900 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424 not finished, retry = 0 2023-08-17 16:56:28,912 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-08-17 16:56:28,913 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-17 16:56:28,914 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:29,003 DEBUG [RS:1;jenkins-hbase4:42469] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs 2023-08-17 16:56:29,004 INFO [RS:1;jenkins-hbase4:42469] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0:(num 1692291379424) 2023-08-17 16:56:29,009 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.WALEntryStream(207): Couldn't get file length information about log hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424, it was closed cleanly currently replicating from: hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424 at position: 15173 2023-08-17 16:56:29,009 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424 2023-08-17 16:56:29,009 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:56:29,010 DEBUG [RS:1;jenkins-hbase4:42469] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs 2023-08-17 16:56:29,010 INFO [RS:1;jenkins-hbase4:42469] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1:(num 1692291380362) 2023-08-17 16:56:29,010 DEBUG [RS:1;jenkins-hbase4:42469] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:29,010 INFO [RS:1;jenkins-hbase4:42469] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:29,010 INFO [RS:1;jenkins-hbase4:42469] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-17 16:56:29,010 INFO [RS:1;jenkins-hbase4:42469] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-17 16:56:29,010 INFO [RS:1;jenkins-hbase4:42469] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-17 16:56:29,010 INFO [RS:1;jenkins-hbase4:42469] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-17 16:56:29,010 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:56:29,011 INFO [RS:1;jenkins-hbase4:42469] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-17 16:56:29,017 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-17 16:56:29,018 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:29,094 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1504): Waiting on 1588230740, a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:29,103 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1.1692291380362 was moved to hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1.1692291380362 2023-08-17 16:56:29,121 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-17 16:56:29,122 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:29,143 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/.tmp/cf1/7f11d4a19b91431ebde61712f73a7bd8 2023-08-17 16:56:29,144 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/.tmp/info/1084509cff0c4142899836b8c21573d3 2023-08-17 16:56:29,149 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7f11d4a19b91431ebde61712f73a7bd8 2023-08-17 16:56:29,154 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410017, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:29,154 DEBUG [RS:1;jenkins-hbase4:42469] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6c9d61ff to 127.0.0.1:56119 2023-08-17 16:56:29,154 DEBUG [RS:1;jenkins-hbase4:42469] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:29,154 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410017, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:29,165 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/.tmp/rep_barrier/9238b01861dd4e1cbed45fe870815754 2023-08-17 16:56:29,179 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/.tmp/table/d480699fbb4445a09cbaf714b1a4e790 2023-08-17 16:56:29,184 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/.tmp/info/1084509cff0c4142899836b8c21573d3 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/info/1084509cff0c4142899836b8c21573d3 2023-08-17 16:56:29,190 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/info/1084509cff0c4142899836b8c21573d3, entries=30, sequenceid=19, filesize=8.1 K 2023-08-17 16:56:29,190 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/.tmp/rep_barrier/9238b01861dd4e1cbed45fe870815754 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/rep_barrier/9238b01861dd4e1cbed45fe870815754 2023-08-17 16:56:29,195 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/rep_barrier/9238b01861dd4e1cbed45fe870815754, entries=2, sequenceid=19, filesize=4.9 K 2023-08-17 16:56:29,196 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/.tmp/table/d480699fbb4445a09cbaf714b1a4e790 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/table/d480699fbb4445a09cbaf714b1a4e790 2023-08-17 16:56:29,201 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/table/d480699fbb4445a09cbaf714b1a4e790, entries=6, sequenceid=19, filesize=4.8 K 2023-08-17 16:56:29,202 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 507ms, sequenceid=19, compaction requested=false 2023-08-17 16:56:29,211 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-08-17 16:56:29,212 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-17 16:56:29,212 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-17 16:56:29,212 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-17 16:56:29,213 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-17 16:56:29,226 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-17 16:56:29,226 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:29,257 INFO [RS:1;jenkins-hbase4:42469] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1 terminated 2023-08-17 16:56:29,258 INFO [RS:1;jenkins-hbase4:42469] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1 terminated 2023-08-17 16:56:29,259 INFO [RS:1;jenkins-hbase4:42469] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42469 2023-08-17 16:56:29,261 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-17 16:56:29,261 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:29,261 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:29,261 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-17 16:56:29,261 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-17 16:56:29,261 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42469,1692291378095] 2023-08-17 16:56:29,261 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42469,1692291378095; numProcessing=1 2023-08-17 16:56:29,267 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42469,1692291378095 already deleted, retry=false 2023-08-17 16:56:29,267 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42469,1692291378095 expired; onlineServers=1 2023-08-17 16:56:29,295 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1504): Waiting on a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:29,303 INFO [jenkins-hbase4:45379Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=8120B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 200, current progress: walGroup [jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1]: currently replicating from: hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118 at position: 21752 walGroup [jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0]: currently replicating from: hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0.1692291379424 at position: -1 2023-08-17 16:56:29,331 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:29,331 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-17 16:56:29,345 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-17 16:56:29,345 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-17 16:56:29,366 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:29,366 INFO [RS:1;jenkins-hbase4:42469] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42469,1692291378095; zookeeper connection closed. 2023-08-17 16:56:29,367 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:42469-0x1001b2c4f410002, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:29,367 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@73304df8] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@73304df8 2023-08-17 16:56:29,436 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:29,436 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-17 16:56:29,495 DEBUG [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1504): Waiting on a2668222e397f37ad3389946a7fe032f 2023-08-17 16:56:29,541 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:29,541 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-17 16:56:29,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/.tmp/norep/4ba41047d6f94283b79849d37770ba50 2023-08-17 16:56:29,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4ba41047d6f94283b79849d37770ba50 2023-08-17 16:56:29,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/.tmp/cf1/7f11d4a19b91431ebde61712f73a7bd8 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/cf1/7f11d4a19b91431ebde61712f73a7bd8 2023-08-17 16:56:29,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7f11d4a19b91431ebde61712f73a7bd8 2023-08-17 16:56:29,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/cf1/7f11d4a19b91431ebde61712f73a7bd8, entries=200, sequenceid=206, filesize=11.9 K 2023-08-17 16:56:29,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/.tmp/norep/4ba41047d6f94283b79849d37770ba50 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/norep/4ba41047d6f94283b79849d37770ba50 2023-08-17 16:56:29,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4ba41047d6f94283b79849d37770ba50 2023-08-17 16:56:29,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/norep/4ba41047d6f94283b79849d37770ba50, entries=101, sequenceid=206, filesize=8.3 K 2023-08-17 16:56:29,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for a2668222e397f37ad3389946a7fe032f in 894ms, sequenceid=206, compaction requested=false 2023-08-17 16:56:29,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-08-17 16:56:29,594 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:29,594 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a2668222e397f37ad3389946a7fe032f: 2023-08-17 16:56:29,594 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:56:29,646 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:29,646 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-08-17 16:56:29,646 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:56:29,646 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-08-17 16:56:29,695 INFO [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45379,1692291378053; all regions closed. 2023-08-17 16:56:29,702 DEBUG [RS:0;jenkins-hbase4:45379] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs 2023-08-17 16:56:29,702 INFO [RS:0;jenkins-hbase4:45379] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45379%2C1692291378053.meta:.meta(num 1692291379605) 2023-08-17 16:56:29,707 DEBUG [RS:0;jenkins-hbase4:45379] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs 2023-08-17 16:56:29,707 INFO [RS:0;jenkins-hbase4:45379] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0:(num 1692291379424) 2023-08-17 16:56:29,712 DEBUG [RS:0;jenkins-hbase4:45379] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs 2023-08-17 16:56:29,712 INFO [RS:0;jenkins-hbase4:45379] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1:(num 1692291384118) 2023-08-17 16:56:29,712 DEBUG [RS:0;jenkins-hbase4:45379] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:29,712 INFO [RS:0;jenkins-hbase4:45379] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:56:29,712 INFO [RS:0;jenkins-hbase4:45379] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-17 16:56:29,712 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:56:29,713 INFO [RS:0;jenkins-hbase4:45379] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-17 16:56:29,747 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0.1692291379424 was moved to hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0.1692291379424 2023-08-17 16:56:29,747 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118 was moved to hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118 2023-08-17 16:56:29,815 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410018, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:29,815 DEBUG [RS:0;jenkins-hbase4:45379] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7c6c2cd4 to 127.0.0.1:56119 2023-08-17 16:56:29,815 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410018, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:29,815 DEBUG [RS:0;jenkins-hbase4:45379] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:29,915 INFO [RS:0;jenkins-hbase4:45379] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1 terminated 2023-08-17 16:56:29,915 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:381) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) 2023-08-17 16:56:29,917 INFO [RS:0;jenkins-hbase4:45379] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1 terminated 2023-08-17 16:56:29,918 INFO [RS:0;jenkins-hbase4:45379] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45379 2023-08-17 16:56:29,921 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:29,921 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-17 16:56:29,922 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45379,1692291378053] 2023-08-17 16:56:29,922 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45379,1692291378053; numProcessing=2 2023-08-17 16:56:29,923 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,45379,1692291378053 already deleted, retry=false 2023-08-17 16:56:29,923 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45379,1692291378053 expired; onlineServers=0 2023-08-17 16:56:29,923 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33151,1692291376904' ***** 2023-08-17 16:56:29,923 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-17 16:56:29,924 DEBUG [M:0;jenkins-hbase4:33151] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@39dcfef5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:56:29,924 INFO [M:0;jenkins-hbase4:33151] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33151,1692291376904 2023-08-17 16:56:29,924 INFO [M:0;jenkins-hbase4:33151] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33151,1692291376904; all regions closed. 2023-08-17 16:56:29,924 DEBUG [M:0;jenkins-hbase4:33151] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:29,924 DEBUG [M:0;jenkins-hbase4:33151] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-17 16:56:29,924 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-17 16:56:29,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291379100] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291379100,5,FailOnTimeoutGroup] 2023-08-17 16:56:29,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291379100] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291379100,5,FailOnTimeoutGroup] 2023-08-17 16:56:29,925 DEBUG [M:0;jenkins-hbase4:33151] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-17 16:56:29,926 INFO [M:0;jenkins-hbase4:33151] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-17 16:56:29,926 INFO [M:0;jenkins-hbase4:33151] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-17 16:56:29,926 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-08-17 16:56:29,927 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:29,927 INFO [M:0;jenkins-hbase4:33151] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-17 16:56:29,927 DEBUG [M:0;jenkins-hbase4:33151] master.HMaster(1512): Stopping service threads 2023-08-17 16:56:29,927 INFO [M:0;jenkins-hbase4:33151] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-17 16:56:29,927 ERROR [M:0;jenkins-hbase4:33151] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-08-17 16:56:29,927 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-17 16:56:29,927 INFO [M:0;jenkins-hbase4:33151] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-17 16:56:29,928 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-17 16:56:29,928 DEBUG [M:0;jenkins-hbase4:33151] zookeeper.ZKUtil(398): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-08-17 16:56:29,928 WARN [M:0;jenkins-hbase4:33151] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-17 16:56:29,928 INFO [M:0;jenkins-hbase4:33151] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-17 16:56:29,929 INFO [M:0;jenkins-hbase4:33151] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-17 16:56:29,929 DEBUG [M:0;jenkins-hbase4:33151] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-17 16:56:29,929 INFO [M:0;jenkins-hbase4:33151] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:29,929 DEBUG [M:0;jenkins-hbase4:33151] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:29,929 DEBUG [M:0;jenkins-hbase4:33151] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-17 16:56:29,929 DEBUG [M:0;jenkins-hbase4:33151] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:29,929 INFO [M:0;jenkins-hbase4:33151] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.05 KB heapSize=72.62 KB 2023-08-17 16:56:29,944 INFO [M:0;jenkins-hbase4:33151] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.05 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/9f129a1dda534bd88a49ad3831664095 2023-08-17 16:56:29,949 DEBUG [M:0;jenkins-hbase4:33151] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/9f129a1dda534bd88a49ad3831664095 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/9f129a1dda534bd88a49ad3831664095 2023-08-17 16:56:29,954 INFO [M:0;jenkins-hbase4:33151] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/9f129a1dda534bd88a49ad3831664095, entries=17, sequenceid=136, filesize=8.5 K 2023-08-17 16:56:29,955 INFO [M:0;jenkins-hbase4:33151] regionserver.HRegion(2948): Finished flush of dataSize ~61.05 KB/62520, heapSize ~72.60 KB/74344, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 25ms, sequenceid=136, compaction requested=false 2023-08-17 16:56:29,956 INFO [M:0;jenkins-hbase4:33151] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:56:29,956 DEBUG [M:0;jenkins-hbase4:33151] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:56:29,959 INFO [M:0;jenkins-hbase4:33151] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-17 16:56:29,959 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:56:29,959 INFO [M:0;jenkins-hbase4:33151] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33151 2023-08-17 16:56:29,962 DEBUG [M:0;jenkins-hbase4:33151] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,33151,1692291376904 already deleted, retry=false 2023-08-17 16:56:30,022 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:30,022 INFO [RS:0;jenkins-hbase4:45379] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45379,1692291378053; zookeeper connection closed. 2023-08-17 16:56:30,022 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): regionserver:45379-0x1001b2c4f410001, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:30,023 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@d4c3d5a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@d4c3d5a 2023-08-17 16:56:30,023 INFO [Listener at localhost/35639] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-08-17 16:56:30,122 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:30,122 INFO [M:0;jenkins-hbase4:33151] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33151,1692291376904; zookeeper connection closed. 2023-08-17 16:56:30,122 DEBUG [Listener at localhost/44847-EventThread] zookeeper.ZKWatcher(600): master:33151-0x1001b2c4f410000, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:30,125 INFO [Listener at localhost/35639] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-08-17 16:56:30,125 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:30,125 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:30,125 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:30,125 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:30,125 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:30,126 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:30,126 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38743 2023-08-17 16:56:30,127 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:30,128 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:30,128 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38743 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:30,132 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:387430x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:30,133 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38743-0x1001b2c4f41001c connected 2023-08-17 16:56:30,135 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:56:30,136 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:30,136 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-17 16:56:30,136 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38743 2023-08-17 16:56:30,137 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38743 2023-08-17 16:56:30,137 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38743 2023-08-17 16:56:30,137 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38743 2023-08-17 16:56:30,137 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38743 2023-08-17 16:56:30,138 INFO [Listener at localhost/35639] master.HMaster(444): hbase.rootdir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4, hbase.cluster.distributed=false 2023-08-17 16:56:30,138 DEBUG [pool-456-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-08-17 16:56:30,151 INFO [Listener at localhost/35639] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-17 16:56:30,152 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:30,152 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:30,152 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:30,152 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:30,152 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:30,152 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:30,153 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34993 2023-08-17 16:56:30,153 INFO [Listener at localhost/35639] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-17 16:56:30,154 DEBUG [Listener at localhost/35639] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-17 16:56:30,155 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:30,156 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:30,157 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34993 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:30,161 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:349930x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:30,163 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34993-0x1001b2c4f41001d connected 2023-08-17 16:56:30,163 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:56:30,163 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:56:30,164 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-17 16:56:30,164 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34993 2023-08-17 16:56:30,164 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34993 2023-08-17 16:56:30,164 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34993 2023-08-17 16:56:30,165 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34993 2023-08-17 16:56:30,166 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34993 2023-08-17 16:56:30,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,168 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-17 16:56:30,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,170 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-17 16:56:30,170 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-17 16:56:30,170 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:30,171 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-17 16:56:30,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,38743,1692291390124 from backup master directory 2023-08-17 16:56:30,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-17 16:56:30,173 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,173 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:30,173 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-17 16:56:30,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:30,213 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x289436ae to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:30,216 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5f288611, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:30,217 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:30,217 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-17 16:56:30,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:30,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:30,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34345,1692291381639 to hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34345,1692291381639-dead as it is dead 2023-08-17 16:56:30,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34345,1692291381639-dead/jenkins-hbase4.apache.org%2C34345%2C1692291381639.jenkins-hbase4.apache.org%2C34345%2C1692291381639.regiongroup-0.1692291381884 2023-08-17 16:56:30,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34345,1692291381639-dead/jenkins-hbase4.apache.org%2C34345%2C1692291381639.jenkins-hbase4.apache.org%2C34345%2C1692291381639.regiongroup-0.1692291381884 after 5ms 2023-08-17 16:56:30,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34345,1692291381639-dead/jenkins-hbase4.apache.org%2C34345%2C1692291381639.jenkins-hbase4.apache.org%2C34345%2C1692291381639.regiongroup-0.1692291381884 to hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34345%2C1692291381639.jenkins-hbase4.apache.org%2C34345%2C1692291381639.regiongroup-0.1692291381884 2023-08-17 16:56:30,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34345,1692291381639-dead 2023-08-17 16:56:30,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,234 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:30,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38743%2C1692291390124.jenkins-hbase4.apache.org%2C38743%2C1692291390124.regiongroup-0, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,38743,1692291390124, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/oldWALs, maxLogs=10 2023-08-17 16:56:30,251 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK] 2023-08-17 16:56:30,251 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:30,251 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:56:30,254 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,38743,1692291390124/jenkins-hbase4.apache.org%2C38743%2C1692291390124.jenkins-hbase4.apache.org%2C38743%2C1692291390124.regiongroup-0.1692291390236 2023-08-17 16:56:30,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK], DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK]] 2023-08-17 16:56:30,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:30,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:30,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:30,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:30,256 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:30,257 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-17 16:56:30,257 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-17 16:56:30,263 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/0c1abd12533b4b5aae99c934fcfdd5b9 2023-08-17 16:56:30,264 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:30,265 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-08-17 16:56:30,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34345%2C1692291381639.jenkins-hbase4.apache.org%2C34345%2C1692291381639.regiongroup-0.1692291381884 2023-08-17 16:56:30,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34345%2C1692291381639.jenkins-hbase4.apache.org%2C34345%2C1692291381639.regiongroup-0.1692291381884 2023-08-17 16:56:30,271 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34345%2C1692291381639.jenkins-hbase4.apache.org%2C34345%2C1692291381639.regiongroup-0.1692291381884 2023-08-17 16:56:30,274 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:30,276 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-08-17 16:56:30,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58993782, jitterRate=-0.12092414498329163}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-17 16:56:30,277 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:56:30,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-17 16:56:30,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-17 16:56:30,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-17 16:56:30,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-17 16:56:30,279 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-17 16:56:30,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-17 16:56:30,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-17 16:56:30,295 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-17 16:56:30,295 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-17 16:56:30,295 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-17 16:56:30,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-08-17 16:56:30,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-08-17 16:56:30,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 16 msec 2023-08-17 16:56:30,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-17 16:56:30,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-17 16:56:30,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38843,1692291381706, table=hbase:meta, region=1588230740 2023-08-17 16:56:30,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-08-17 16:56:30,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,38799,1692291381679 already deleted, retry=false 2023-08-17 16:56:30,302 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,38799,1692291381679 on jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,38799,1692291381679, splitWal=true, meta=false 2023-08-17 16:56:30,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,38799,1692291381679 (carryingMeta=false) jenkins-hbase4.apache.org,38799,1692291381679/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@2a8d9c3a[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-17 16:56:30,312 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,38843,1692291381706 already deleted, retry=false 2023-08-17 16:56:30,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,38843,1692291381706 on jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,313 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,38843,1692291381706, splitWal=true, meta=true 2023-08-17 16:56:30,313 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase4.apache.org,38843,1692291381706 (carryingMeta=true) jenkins-hbase4.apache.org,38843,1692291381706/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@5aea8c7d[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-17 16:56:30,316 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,33525,1692291381734 already deleted, retry=false 2023-08-17 16:56:30,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,33525,1692291381734 on jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,317 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,33525,1692291381734, splitWal=true, meta=false 2023-08-17 16:56:30,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase4.apache.org,33525,1692291381734 (carryingMeta=false) jenkins-hbase4.apache.org,33525,1692291381734/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@1c40b756[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-17 16:56:30,318 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,41501,1692291381758 already deleted, retry=false 2023-08-17 16:56:30,318 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,41501,1692291381758 on jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,319 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,41501,1692291381758, splitWal=true, meta=false 2023-08-17 16:56:30,319 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,41501,1692291381758 (carryingMeta=false) jenkins-hbase4.apache.org,41501,1692291381758/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@32019cf0[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-17 16:56:30,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-17 16:56:30,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-17 16:56:30,330 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-08-17 16:56:30,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-17 16:56:30,330 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-08-17 16:56:30,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-08-17 16:56:30,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-08-17 16:56:30,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-08-17 16:56:30,334 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-17 16:56:30,334 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-17 16:56:30,335 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:56:30,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38743,1692291390124, sessionid=0x1001b2c4f41001c, setting cluster-up flag (Was=false) 2023-08-17 16:56:30,340 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-08-17 16:56:30,341 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-08-17 16:56:30,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:56:30,350 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.hbase-snapshot/.tmp 2023-08-17 16:56:30,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:30,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:30,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:30,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:30,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-17 16:56:30,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:30,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,362 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692291420362 2023-08-17 16:56:30,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-17 16:56:30,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-17 16:56:30,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-17 16:56:30,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-17 16:56:30,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-17 16:56:30,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-17 16:56:30,372 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:30,374 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33525,1692291381734; numProcessing=1 2023-08-17 16:56:30,374 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38843,1692291381706; numProcessing=2 2023-08-17 16:56:30,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-17 16:56:30,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-17 16:56:30,375 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-17 16:56:30,375 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41501,1692291381758; numProcessing=3 2023-08-17 16:56:30,375 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38843,1692291381706, splitWal=true, meta=true 2023-08-17 16:56:30,375 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,41501,1692291381758, splitWal=true, meta=false 2023-08-17 16:56:30,375 DEBUG [PEWorker-5] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38799,1692291381679; numProcessing=4 2023-08-17 16:56:30,375 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33525,1692291381734, splitWal=true, meta=false 2023-08-17 16:56:30,375 INFO [PEWorker-5] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38799,1692291381679, splitWal=true, meta=false 2023-08-17 16:56:30,376 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(951): ClusterId : 54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:30,376 DEBUG [RS:0;jenkins-hbase4:34993] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-17 16:56:30,377 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38843,1692291381706, splitWal=true, meta=true, isMeta: true 2023-08-17 16:56:30,378 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706-splitting 2023-08-17 16:56:30,379 DEBUG [RS:0;jenkins-hbase4:34993] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-17 16:56:30,379 DEBUG [RS:0;jenkins-hbase4:34993] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-17 16:56:30,379 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706-splitting dir is empty, no logs to split. 2023-08-17 16:56:30,379 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,38843,1692291381706 WAL count=0, meta=true 2023-08-17 16:56:30,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-17 16:56:30,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-17 16:56:30,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291390387,5,FailOnTimeoutGroup] 2023-08-17 16:56:30,387 DEBUG [RS:0;jenkins-hbase4:34993] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-17 16:56:30,389 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291390387,5,FailOnTimeoutGroup] 2023-08-17 16:56:30,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:30,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-17 16:56:30,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:30,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:30,390 DEBUG [RS:0;jenkins-hbase4:34993] zookeeper.ReadOnlyZKClient(139): Connect 0x5724d046 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:30,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692291390390, completionTime=-1 2023-08-17 16:56:30,390 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-08-17 16:56:30,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-17 16:56:30,392 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706-splitting dir is empty, no logs to split. 2023-08-17 16:56:30,392 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,38843,1692291381706 WAL count=0, meta=true 2023-08-17 16:56:30,392 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,38843,1692291381706 WAL splitting is done? wals=0, meta=true 2023-08-17 16:56:30,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-17 16:56:30,396 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-17 16:56:30,397 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:56:30,397 DEBUG [RS:0;jenkins-hbase4:34993] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@59e334d2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:30,397 DEBUG [RS:0;jenkins-hbase4:34993] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ddcbb36, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:56:30,406 DEBUG [RS:0;jenkins-hbase4:34993] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:34993 2023-08-17 16:56:30,406 INFO [RS:0;jenkins-hbase4:34993] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-17 16:56:30,406 INFO [RS:0;jenkins-hbase4:34993] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-17 16:56:30,406 DEBUG [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1022): About to register with Master. 2023-08-17 16:56:30,407 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38743,1692291390124 with isa=jenkins-hbase4.apache.org/172.31.14.131:34993, startcode=1692291390151 2023-08-17 16:56:30,407 DEBUG [RS:0;jenkins-hbase4:34993] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-17 16:56:30,409 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57923, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-08-17 16:56:30,410 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38743] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:30,410 DEBUG [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4 2023-08-17 16:56:30,410 DEBUG [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41465 2023-08-17 16:56:30,410 DEBUG [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-17 16:56:30,416 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:56:30,416 DEBUG [RS:0;jenkins-hbase4:34993] zookeeper.ZKUtil(162): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:30,417 WARN [RS:0;jenkins-hbase4:34993] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:30,417 INFO [RS:0;jenkins-hbase4:34993] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:30,417 INFO [RS:0;jenkins-hbase4:34993] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:30,417 DEBUG [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:30,418 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34993,1692291390151] 2023-08-17 16:56:30,421 DEBUG [RS:0;jenkins-hbase4:34993] zookeeper.ZKUtil(162): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:30,422 DEBUG [RS:0;jenkins-hbase4:34993] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:56:30,422 INFO [RS:0;jenkins-hbase4:34993] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-17 16:56:30,423 INFO [RS:0;jenkins-hbase4:34993] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-17 16:56:30,424 INFO [RS:0;jenkins-hbase4:34993] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-17 16:56:30,424 INFO [RS:0;jenkins-hbase4:34993] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:30,424 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-17 16:56:30,426 INFO [RS:0;jenkins-hbase4:34993] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:30,426 DEBUG [RS:0;jenkins-hbase4:34993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,426 DEBUG [RS:0;jenkins-hbase4:34993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,426 DEBUG [RS:0;jenkins-hbase4:34993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,426 DEBUG [RS:0;jenkins-hbase4:34993] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,426 DEBUG [RS:0;jenkins-hbase4:34993] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,426 DEBUG [RS:0;jenkins-hbase4:34993] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:30,426 DEBUG [RS:0;jenkins-hbase4:34993] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,426 DEBUG [RS:0;jenkins-hbase4:34993] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,426 DEBUG [RS:0;jenkins-hbase4:34993] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,427 DEBUG [RS:0;jenkins-hbase4:34993] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:30,429 INFO [RS:0;jenkins-hbase4:34993] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:30,429 INFO [RS:0;jenkins-hbase4:34993] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:30,430 INFO [RS:0;jenkins-hbase4:34993] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:30,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-17 16:56:30,441 INFO [RS:0;jenkins-hbase4:34993] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-17 16:56:30,441 INFO [RS:0;jenkins-hbase4:34993] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34993,1692291390151-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:30,454 INFO [RS:0;jenkins-hbase4:34993] regionserver.Replication(203): jenkins-hbase4.apache.org,34993,1692291390151 started 2023-08-17 16:56:30,455 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34993,1692291390151, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34993, sessionid=0x1001b2c4f41001d 2023-08-17 16:56:30,455 DEBUG [RS:0;jenkins-hbase4:34993] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-17 16:56:30,455 DEBUG [RS:0;jenkins-hbase4:34993] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:30,455 DEBUG [RS:0;jenkins-hbase4:34993] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34993,1692291390151' 2023-08-17 16:56:30,455 DEBUG [RS:0;jenkins-hbase4:34993] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-17 16:56:30,455 DEBUG [RS:0;jenkins-hbase4:34993] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-17 16:56:30,456 DEBUG [RS:0;jenkins-hbase4:34993] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-17 16:56:30,456 DEBUG [RS:0;jenkins-hbase4:34993] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-17 16:56:30,456 DEBUG [RS:0;jenkins-hbase4:34993] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:30,456 DEBUG [RS:0;jenkins-hbase4:34993] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34993,1692291390151' 2023-08-17 16:56:30,456 DEBUG [RS:0;jenkins-hbase4:34993] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-17 16:56:30,456 DEBUG [RS:0;jenkins-hbase4:34993] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-17 16:56:30,456 DEBUG [RS:0;jenkins-hbase4:34993] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-17 16:56:30,456 INFO [RS:0;jenkins-hbase4:34993] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-17 16:56:30,456 INFO [RS:0;jenkins-hbase4:34993] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-17 16:56:30,547 DEBUG [jenkins-hbase4:38743] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-17 16:56:30,549 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34993,1692291390151, state=OPENING 2023-08-17 16:56:30,550 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-17 16:56:30,550 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-17 16:56:30,550 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34993,1692291390151}] 2023-08-17 16:56:30,557 INFO [RS:0;jenkins-hbase4:34993] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:30,559 INFO [RS:0;jenkins-hbase4:34993] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34993%2C1692291390151.jenkins-hbase4.apache.org%2C34993%2C1692291390151.regiongroup-0, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:56:30,575 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:56:30,575 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:30,575 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK] 2023-08-17 16:56:30,577 INFO [RS:0;jenkins-hbase4:34993] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151/jenkins-hbase4.apache.org%2C34993%2C1692291390151.jenkins-hbase4.apache.org%2C34993%2C1692291390151.regiongroup-0.1692291390559 2023-08-17 16:56:30,577 DEBUG [RS:0;jenkins-hbase4:34993] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK], DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK]] 2023-08-17 16:56:30,704 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:30,705 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:30,706 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40722, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-17 16:56:30,710 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-17 16:56:30,710 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:30,710 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:30,710 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:30,711 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34993%2C1692291390151.meta, suffix=.meta, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:56:30,727 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:56:30,727 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:30,727 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:56:30,729 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151/jenkins-hbase4.apache.org%2C34993%2C1692291390151.meta.1692291390712.meta 2023-08-17 16:56:30,729 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK], DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK]] 2023-08-17 16:56:30,729 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:30,729 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-17 16:56:30,730 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-17 16:56:30,731 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-17 16:56:30,731 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-17 16:56:30,731 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:30,731 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-17 16:56:30,731 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-17 16:56:30,732 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-17 16:56:30,733 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info 2023-08-17 16:56:30,733 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info 2023-08-17 16:56:30,733 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-17 16:56:30,739 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info/d862d25bca50496b8832b846a15b761e 2023-08-17 16:56:30,739 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:30,739 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-17 16:56:30,740 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:56:30,740 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:56:30,741 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-17 16:56:30,741 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:30,742 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-17 16:56:30,742 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table 2023-08-17 16:56:30,742 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table 2023-08-17 16:56:30,743 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-17 16:56:30,748 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table/0a63d2c9dbf3474c8afb858114c8a01a 2023-08-17 16:56:30,748 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:30,749 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740 2023-08-17 16:56:30,750 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740 2023-08-17 16:56:30,753 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-17 16:56:30,754 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-17 16:56:30,755 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73410171, jitterRate=0.0938967913389206}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-17 16:56:30,755 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-17 16:56:30,756 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1692291390704 2023-08-17 16:56:30,760 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-17 16:56:30,761 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-17 16:56:30,761 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34993,1692291390151, state=OPEN 2023-08-17 16:56:30,762 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-17 16:56:30,763 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-17 16:56:30,764 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-08-17 16:56:30,764 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34993,1692291390151 in 213 msec 2023-08-17 16:56:30,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=16 2023-08-17 16:56:30,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 371 msec 2023-08-17 16:56:31,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1552ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-08-17 16:56:33,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-08-17 16:56:34,897 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-08-17 16:56:34,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-17 16:56:34,901 DEBUG [hconnection-0x35bec3fe-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:34,903 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40738, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:34,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=37b9df605a0d48d3a0fcac9d8ab4c0c4, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38799,1692291381679, regionLocation=jenkins-hbase4.apache.org,38799,1692291381679, openSeqNum=2 2023-08-17 16:56:34,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=7b7c5dd31d075846c031f622b5c28bc1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38843,1692291381706, regionLocation=jenkins-hbase4.apache.org,38843,1692291381706, openSeqNum=2 2023-08-17 16:56:34,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ec395d682a88bce1d4eae00d5f8e72c7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38799,1692291381679, regionLocation=jenkins-hbase4.apache.org,38799,1692291381679, openSeqNum=2 2023-08-17 16:56:34,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-17 16:56:34,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692291454912 2023-08-17 16:56:34,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692291514912 2023-08-17 16:56:34,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 15 msec 2023-08-17 16:56:34,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38743,1692291390124-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:34,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38743,1692291390124-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:34,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38743,1692291390124-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:34,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38743, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:34,936 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,33525,1692291381734 had 0 regions 2023-08-17 16:56:34,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:34,936 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,38843,1692291381706 had 2 regions 2023-08-17 16:56:34,936 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,38799,1692291381679 had 2 regions 2023-08-17 16:56:34,936 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,41501,1692291381758 had 0 regions 2023-08-17 16:56:34,937 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. is NOT online; state={37b9df605a0d48d3a0fcac9d8ab4c0c4 state=OPEN, ts=1692291394912, server=jenkins-hbase4.apache.org,38799,1692291381679}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-08-17 16:56:34,938 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,41501,1692291381758, splitWal=true, meta=false, isMeta: false 2023-08-17 16:56:34,938 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38799,1692291381679, splitWal=true, meta=false, isMeta: false 2023-08-17 16:56:34,938 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38843,1692291381706, splitWal=true, meta=true, isMeta: false 2023-08-17 16:56:34,938 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33525,1692291381734, splitWal=true, meta=false, isMeta: false 2023-08-17 16:56:34,939 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,41501,1692291381758-splitting 2023-08-17 16:56:34,941 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,41501,1692291381758-splitting dir is empty, no logs to split. 2023-08-17 16:56:34,941 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,41501,1692291381758 WAL count=0, meta=false 2023-08-17 16:56:34,941 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38799,1692291381679-splitting 2023-08-17 16:56:34,942 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38799,1692291381679-splitting dir is empty, no logs to split. 2023-08-17 16:56:34,942 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,38799,1692291381679 WAL count=0, meta=false 2023-08-17 16:56:34,943 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,38799,1692291381679/hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4., unknown_server=jenkins-hbase4.apache.org,38843,1692291381706/t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1., unknown_server=jenkins-hbase4.apache.org,38799,1692291381679/t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:34,944 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,33525,1692291381734-splitting 2023-08-17 16:56:34,944 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706-splitting dir is empty, no logs to split. 2023-08-17 16:56:34,944 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,38843,1692291381706 WAL count=0, meta=false 2023-08-17 16:56:34,944 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,33525,1692291381734-splitting dir is empty, no logs to split. 2023-08-17 16:56:34,945 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,33525,1692291381734 WAL count=0, meta=false 2023-08-17 16:56:34,946 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,41501,1692291381758-splitting dir is empty, no logs to split. 2023-08-17 16:56:34,946 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,41501,1692291381758 WAL count=0, meta=false 2023-08-17 16:56:34,946 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,41501,1692291381758 WAL splitting is done? wals=0, meta=false 2023-08-17 16:56:34,946 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38799,1692291381679-splitting dir is empty, no logs to split. 2023-08-17 16:56:34,946 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,38799,1692291381679 WAL count=0, meta=false 2023-08-17 16:56:34,947 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,38799,1692291381679 WAL splitting is done? wals=0, meta=false 2023-08-17 16:56:34,947 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38843,1692291381706-splitting dir is empty, no logs to split. 2023-08-17 16:56:34,947 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,38843,1692291381706 WAL count=0, meta=false 2023-08-17 16:56:34,947 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,38843,1692291381706 WAL splitting is done? wals=0, meta=false 2023-08-17 16:56:34,948 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,33525,1692291381734-splitting dir is empty, no logs to split. 2023-08-17 16:56:34,949 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,33525,1692291381734 WAL count=0, meta=false 2023-08-17 16:56:34,949 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,41501,1692291381758 failed, ignore...File hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,41501,1692291381758-splitting does not exist. 2023-08-17 16:56:34,949 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,33525,1692291381734 WAL splitting is done? wals=0, meta=false 2023-08-17 16:56:34,949 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,38799,1692291381679 failed, ignore...File hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,38799,1692291381679-splitting does not exist. 2023-08-17 16:56:34,949 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN}] 2023-08-17 16:56:34,950 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN}, {pid=23, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN}] 2023-08-17 16:56:34,951 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,41501,1692291381758 after splitting done 2023-08-17 16:56:34,951 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,41501,1692291381758 from processing; numProcessing=3 2023-08-17 16:56:34,951 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN 2023-08-17 16:56:34,951 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN 2023-08-17 16:56:34,951 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN 2023-08-17 16:56:34,952 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:56:34,952 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,33525,1692291381734 failed, ignore...File hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,33525,1692291381734-splitting does not exist. 2023-08-17 16:56:34,952 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:56:34,953 DEBUG [jenkins-hbase4:38743] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-17 16:56:34,954 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:56:34,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,41501,1692291381758, splitWal=true, meta=false in 4.6330 sec 2023-08-17 16:56:34,955 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=37b9df605a0d48d3a0fcac9d8ab4c0c4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:34,955 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291394955"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291394955"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291394955"}]},"ts":"1692291394955"} 2023-08-17 16:56:34,955 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7b7c5dd31d075846c031f622b5c28bc1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:34,955 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291394955"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291394955"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291394955"}]},"ts":"1692291394955"} 2023-08-17 16:56:34,956 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,33525,1692291381734 after splitting done 2023-08-17 16:56:34,956 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,33525,1692291381734 from processing; numProcessing=2 2023-08-17 16:56:34,957 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33525,1692291381734, splitWal=true, meta=false in 4.6400 sec 2023-08-17 16:56:34,958 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=22, state=RUNNABLE; OpenRegionProcedure 37b9df605a0d48d3a0fcac9d8ab4c0c4, server=jenkins-hbase4.apache.org,34993,1692291390151}] 2023-08-17 16:56:34,959 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=21, state=RUNNABLE; OpenRegionProcedure 7b7c5dd31d075846c031f622b5c28bc1, server=jenkins-hbase4.apache.org,34993,1692291390151}] 2023-08-17 16:56:35,106 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=ec395d682a88bce1d4eae00d5f8e72c7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:35,106 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291395105"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291395105"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291395105"}]},"ts":"1692291395105"} 2023-08-17 16:56:35,108 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure ec395d682a88bce1d4eae00d5f8e72c7, server=jenkins-hbase4.apache.org,34993,1692291390151}] 2023-08-17 16:56:35,114 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:35,114 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:35,115 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34993%2C1692291390151.jenkins-hbase4.apache.org%2C34993%2C1692291390151.regiongroup-1, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:56:35,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:35,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ec395d682a88bce1d4eae00d5f8e72c7, NAME => 't2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:35,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:35,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:35,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:35,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:35,118 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:35,119 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1 2023-08-17 16:56:35,119 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1 2023-08-17 16:56:35,120 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec395d682a88bce1d4eae00d5f8e72c7 columnFamilyName cf1 2023-08-17 16:56:35,125 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1/5a8be9b14e044fb4be92342ffecc0175 2023-08-17 16:56:35,125 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(310): Store=ec395d682a88bce1d4eae00d5f8e72c7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:35,125 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:35,126 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/norep 2023-08-17 16:56:35,126 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/norep 2023-08-17 16:56:35,127 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec395d682a88bce1d4eae00d5f8e72c7 columnFamilyName norep 2023-08-17 16:56:35,127 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(310): Store=ec395d682a88bce1d4eae00d5f8e72c7/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:35,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:35,129 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:56:35,129 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:56:35,129 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:56:35,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:35,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-17 16:56:35,133 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151/jenkins-hbase4.apache.org%2C34993%2C1692291390151.jenkins-hbase4.apache.org%2C34993%2C1692291390151.regiongroup-1.1692291395115 2023-08-17 16:56:35,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK], DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK]] 2023-08-17 16:56:35,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 37b9df605a0d48d3a0fcac9d8ab4c0c4, NAME => 'hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:35,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:35,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:35,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:35,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:35,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:56:35,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ec395d682a88bce1d4eae00d5f8e72c7; next sequenceid=12; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74810061, jitterRate=0.11475677788257599}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-17 16:56:35,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ec395d682a88bce1d4eae00d5f8e72c7: 2023-08-17 16:56:35,135 INFO [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:35,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7., pid=26, masterSystemTime=1692291395110 2023-08-17 16:56:35,136 DEBUG [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/info 2023-08-17 16:56:35,136 DEBUG [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/info 2023-08-17 16:56:35,137 INFO [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 37b9df605a0d48d3a0fcac9d8ab4c0c4 columnFamilyName info 2023-08-17 16:56:35,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:35,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:56:35,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:35,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7b7c5dd31d075846c031f622b5c28bc1, NAME => 't1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:35,139 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=ec395d682a88bce1d4eae00d5f8e72c7, regionState=OPEN, openSeqNum=12, regionLocation=jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:35,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:35,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:35,139 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291395139"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291395139"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291395139"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291395139"}]},"ts":"1692291395139"} 2023-08-17 16:56:35,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:35,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:35,140 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:35,141 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1 2023-08-17 16:56:35,141 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1 2023-08-17 16:56:35,142 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b7c5dd31d075846c031f622b5c28bc1 columnFamilyName cf1 2023-08-17 16:56:35,143 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-08-17 16:56:35,143 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure ec395d682a88bce1d4eae00d5f8e72c7, server=jenkins-hbase4.apache.org,34993,1692291390151 in 33 msec 2023-08-17 16:56:35,143 DEBUG [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/info/8bfbb4041ee748bc8dba5ffabe3bdbca 2023-08-17 16:56:35,143 INFO [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] regionserver.HStore(310): Store=37b9df605a0d48d3a0fcac9d8ab4c0c4/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:35,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN in 193 msec 2023-08-17 16:56:35,144 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:35,145 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:35,147 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1/36b4336e2c014c019e4c7d9f163363d2 2023-08-17 16:56:35,147 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(310): Store=7b7c5dd31d075846c031f622b5c28bc1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:35,147 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:35,148 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/norep 2023-08-17 16:56:35,148 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/norep 2023-08-17 16:56:35,149 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:56:35,149 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b7c5dd31d075846c031f622b5c28bc1 columnFamilyName norep 2023-08-17 16:56:35,149 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(310): Store=7b7c5dd31d075846c031f622b5c28bc1/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:35,150 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 37b9df605a0d48d3a0fcac9d8ab4c0c4; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74645977, jitterRate=0.11231173574924469}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-17 16:56:35,150 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 37b9df605a0d48d3a0fcac9d8ab4c0c4: 2023-08-17 16:56:35,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:35,150 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4., pid=24, masterSystemTime=1692291395110 2023-08-17 16:56:35,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:35,152 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:35,152 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:56:35,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-17 16:56:35,154 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=37b9df605a0d48d3a0fcac9d8ab4c0c4, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:35,154 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291395154"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291395154"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291395154"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291395154"}]},"ts":"1692291395154"} 2023-08-17 16:56:35,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:56:35,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7b7c5dd31d075846c031f622b5c28bc1; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64285351, jitterRate=-0.04207362234592438}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-17 16:56:35,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7b7c5dd31d075846c031f622b5c28bc1: 2023-08-17 16:56:35,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1., pid=25, masterSystemTime=1692291395110 2023-08-17 16:56:35,157 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=22 2023-08-17 16:56:35,157 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=22, state=SUCCESS; OpenRegionProcedure 37b9df605a0d48d3a0fcac9d8ab4c0c4, server=jenkins-hbase4.apache.org,34993,1692291390151 in 198 msec 2023-08-17 16:56:35,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:35,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:56:35,158 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7b7c5dd31d075846c031f622b5c28bc1, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:56:35,158 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291395158"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291395158"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291395158"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291395158"}]},"ts":"1692291395158"} 2023-08-17 16:56:35,159 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=15 2023-08-17 16:56:35,159 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,38799,1692291381679 after splitting done 2023-08-17 16:56:35,159 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN in 207 msec 2023-08-17 16:56:35,160 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,38799,1692291381679 from processing; numProcessing=1 2023-08-17 16:56:35,161 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,38799,1692291381679, splitWal=true, meta=false in 4.8570 sec 2023-08-17 16:56:35,161 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=21 2023-08-17 16:56:35,161 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=21, state=SUCCESS; OpenRegionProcedure 7b7c5dd31d075846c031f622b5c28bc1, server=jenkins-hbase4.apache.org,34993,1692291390151 in 200 msec 2023-08-17 16:56:35,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=16 2023-08-17 16:56:35,165 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,38843,1692291381706 after splitting done 2023-08-17 16:56:35,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN in 212 msec 2023-08-17 16:56:35,165 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,38843,1692291381706 from processing; numProcessing=0 2023-08-17 16:56:35,166 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,38843,1692291381706, splitWal=true, meta=true in 4.8530 sec 2023-08-17 16:56:35,420 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-17 16:56:35,454 INFO [jenkins-hbase4:34993Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:35,938 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-08-17 16:56:35,953 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-08-17 16:56:35,955 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-08-17 16:56:35,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.782sec 2023-08-17 16:56:35,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-17 16:56:35,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-17 16:56:35,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-17 16:56:35,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38743,1692291390124-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-17 16:56:35,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38743,1692291390124-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-17 16:56:35,959 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-17 16:56:35,978 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(139): Connect 0x6dc8d824 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:35,983 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e2f64f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:35,984 DEBUG [hconnection-0x7022a50-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:35,986 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42998, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:35,990 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-08-17 16:56:35,990 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6dc8d824 to 127.0.0.1:56119 2023-08-17 16:56:35,990 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:35,991 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:38743 after: jenkins-hbase4.apache.org:38743 2023-08-17 16:56:35,992 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(139): Connect 0x210e00dc to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:35,996 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3748ee07, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:36,423 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-08-17 16:56:36,423 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-17 16:56:36,424 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-17 16:56:36,424 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-08-17 16:56:36,498 DEBUG [hconnection-0x571250c9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:36,500 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43006, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:56:36,522 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1692291396522 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:36,525 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication16922913965220x0, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:36,526 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1692291396522-0x1001b2c4f410022 connected Start Replication Server start 2023-08-17 16:56:36,528 INFO [Listener at localhost/35639] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:36,528 INFO [Listener at localhost/35639] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:36,540 INFO [Listener at localhost/35639] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-17 16:56:36,542 DEBUG [Listener at localhost/35639] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:56:36,552 INFO [Listener at localhost/35639] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-17 16:56:36,553 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,42469,1692291378095, jenkins-hbase4.apache.org,45379,1692291378053] other RSs: [] 2023-08-17 16:56:36,572 DEBUG [Listener at localhost/35639.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x58aa5505 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:36,577 DEBUG [Listener at localhost/35639.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@e4bb877, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:36,578 INFO [Listener at localhost/35639.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:36,582 DEBUG [Listener at localhost/35639.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:36,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1001b2c4f410024 connected 2023-08-17 16:56:36,583 INFO [Listener at localhost/35639.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=1067016f-f973-4161-8f76-366ce33cec43 to cluster=54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:40,415 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,42469,1692291378095/1's WALs to 1692291396528.syncuptool.replication.org,1234,1 2023-08-17 16:56:40,417 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1.1692291380362 with data PBUF\x08\xCD\x06 2023-08-17 16:56:40,417 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424 with data PBUF\x08\xDFU 2023-08-17 16:56:40,419 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,42469,1692291378095/1's WALs to 1692291396528.syncuptool.replication.org,1234,1 2023-08-17 16:56:40,454 INFO [jenkins-hbase4:34993Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:40,620 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-17 16:56:42,421 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42469,1692291378095] 2023-08-17 16:56:42,422 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42469,1692291378095] 2023-08-17 16:56:42,435 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42469,1692291378095] 2023-08-17 16:56:42,435 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,42469,1692291378095, ReplicationSource: 1, currentBandwidth=0 2023-08-17 16:56:42,435 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:42,455 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095] zookeeper.ReadOnlyZKClient(139): Connect 0x460e10e6 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:42,462 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@65c05c53, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:42,462 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:42,466 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:42,467 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1001b2c4f410026 connected 2023-08-17 16:56:42,467 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,42469,1692291378095 (queues=2) is replicating from cluster=1067016f-f973-4161-8f76-366ce33cec43 to cluster=54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:42,468 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0 2023-08-17 16:56:42,471 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42469,1692291378095, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:56:42,472 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1 2023-08-17 16:56:42,472 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0 2023-08-17 16:56:42,473 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42469,1692291378095, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:56:42,474 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1 2023-08-17 16:56:42,480 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1.1692291380362 2023-08-17 16:56:42,480 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424 2023-08-17 16:56:42,480 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:56:42,480 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-17 16:56:42,480 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-17 16:56:42,480 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42469,1692291378095 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1.1692291380362, lastWalPosition=1430, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-08-17 16:56:42,480 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-17 16:56:42,482 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42469,1692291378095] 2023-08-17 16:56:42,482 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42469,1692291378095 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4b04d98f902897a06d2d5c032dc77d72/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424, lastWalPosition=15173, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-08-17 16:56:42,484 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1.1692291380362] 2023-08-17 16:56:42,485 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:42,487 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43014, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-17 16:56:42,488 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34993] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-17 16:56:42,488 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34993] zookeeper.ReadOnlyZKClient(139): Connect 0x44198932 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:42,492 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42469,1692291378095 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:56:42,492 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:42,493 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34993] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4ade7fc1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:42,495 DEBUG [hconnection-0x4ee1e242-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:56:42,496 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43020, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-08-17 16:56:42,506 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34993] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-17 16:56:42,506 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42469,1692291378095] 2023-08-17 16:56:42,507 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0.1692291379424] 2023-08-17 16:56:42,509 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42469,1692291378095 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:56:42,509 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:42,592 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:42,596 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication1692291396522-0x1001b2c4f410022, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1692291396528.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:42,596 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-1,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42469,1692291378095 with the following stats: Total replicated edits: 1, current progress: 2023-08-17 16:56:42,696 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42469,1692291378095 2023-08-17 16:56:42,696 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] zookeeper.ZKUtil(243): syncupReplication1692291396522-0x1001b2c4f410022, quorum=127.0.0.1:56119, baseZNode=/1 Unable to list children of znode /1/replication/rs/1692291396528.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42469,1692291378095 because node does not exist (not an error) 2023-08-17 16:56:42,697 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095.replicationSource.shipperjenkins-hbase4.apache.org%2C42469%2C1692291378095.jenkins-hbase4.apache.org%2C42469%2C1692291378095.regiongroup-0,1-jenkins-hbase4.apache.org,42469,1692291378095] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42469,1692291378095 with the following stats: Total replicated edits: 1, current progress: 2023-08-17 16:56:45,454 INFO [jenkins-hbase4:34993Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:45,455 INFO [jenkins-hbase4:34993Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 1 2023-08-17 16:56:46,064 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,45379,1692291378053/1's WALs to 1692291396528.syncuptool.replication.org,1234,1 2023-08-17 16:56:46,066 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0.1692291379424 with data 2023-08-17 16:56:46,066 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118 with data PBUF\x08\xF8\xA9\x01 2023-08-17 16:56:46,068 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,45379,1692291378053/1's WALs to 1692291396528.syncuptool.replication.org,1234,1 2023-08-17 16:56:48,070 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45379,1692291378053] 2023-08-17 16:56:48,070 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45379,1692291378053] 2023-08-17 16:56:48,081 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45379,1692291378053] 2023-08-17 16:56:48,081 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,45379,1692291378053, ReplicationSource: 1, currentBandwidth=0 2023-08-17 16:56:48,081 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:48,095 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-17 16:56:48,101 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053] zookeeper.ReadOnlyZKClient(139): Connect 0x62c3242c to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:48,105 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1dd76d8e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:48,105 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:48,108 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:48,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1001b2c4f410029 connected 2023-08-17 16:56:48,110 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,45379,1692291378053 (queues=2) is replicating from cluster=1067016f-f973-4161-8f76-366ce33cec43 to cluster=54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:48,110 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1 2023-08-17 16:56:48,111 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,45379,1692291378053, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:56:48,111 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0 2023-08-17 16:56:48,111 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1 2023-08-17 16:56:48,112 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,45379,1692291378053, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:56:48,114 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0 2023-08-17 16:56:48,121 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0.1692291379424 2023-08-17 16:56:48,121 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-17 16:56:48,121 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45379,1692291378053 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:56:48,122 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:48,123 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118 2023-08-17 16:56:48,123 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-17 16:56:48,123 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-17 16:56:48,124 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45379,1692291378053 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/a2668222e397f37ad3389946a7fe032f/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118, lastWalPosition=29258, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-08-17 16:56:48,127 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:48,130 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60686, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-17 16:56:48,131 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34993] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-17 16:56:48,144 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34993] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-17 16:56:48,145 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45379,1692291378053] 2023-08-17 16:56:48,146 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1.1692291384118] 2023-08-17 16:56:48,148 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45379,1692291378053 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:56:48,148 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:48,222 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:48,223 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1-jenkins-hbase4.apache.org,45379,1692291378053] zookeeper.ZKUtil(162): syncupReplication1692291396522-0x1001b2c4f410022, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1692291396528.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0.1692291379424 2023-08-17 16:56:48,225 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication1692291396522-0x1001b2c4f410022, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1692291396528.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,45379,1692291378053/jenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0.1692291379424 2023-08-17 16:56:48,225 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication1692291396522-0x1001b2c4f410022, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1692291396528.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:48,225 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-0,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,45379,1692291378053 with the following stats: Total replicated edits: 1, current progress: 2023-08-17 16:56:48,225 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication1692291396522-0x1001b2c4f410022, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1692291396528.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:48,325 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,45379,1692291378053 2023-08-17 16:56:48,326 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] zookeeper.ZKUtil(243): syncupReplication1692291396522-0x1001b2c4f410022, quorum=127.0.0.1:56119, baseZNode=/1 Unable to list children of znode /1/replication/rs/1692291396528.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,45379,1692291378053 because node does not exist (not an error) 2023-08-17 16:56:48,326 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053.replicationSource.shipperjenkins-hbase4.apache.org%2C45379%2C1692291378053.jenkins-hbase4.apache.org%2C45379%2C1692291378053.regiongroup-1,1-jenkins-hbase4.apache.org,45379,1692291378053] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,45379,1692291378053 with the following stats: Total replicated edits: 1, current progress: 2023-08-17 16:56:50,454 INFO [jenkins-hbase4:34993Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:50,455 INFO [jenkins-hbase4:34993Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-08-17 16:56:53,725 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-17 16:56:55,454 INFO [jenkins-hbase4:34993Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:56:55,455 INFO [jenkins-hbase4:34993Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-08-17 16:56:56,555 INFO [Listener at localhost/35639] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-17 16:56:56,658 DEBUG [Listener at localhost/35639.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410024, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:56,658 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x58aa5505 to 127.0.0.1:56119 2023-08-17 16:56:56,659 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:56:56,861 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication1692291396522-0x1001b2c4f410022, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:56,861 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication1692291396522-0x1001b2c4f410022, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:56:56,874 INFO [Listener at localhost/35639] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-08-17 16:56:56,874 DEBUG [Listener at localhost/35639] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-08-17 16:56:56,876 INFO [Listener at localhost/35639] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-17 16:56:56,876 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:56,876 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:56,876 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:56,876 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:56,877 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:56,877 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:56,877 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42619 2023-08-17 16:56:56,878 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:56,879 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:56,881 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=master:42619 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:56,883 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:426190x0, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:56,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:42619-0x1001b2c4f41002a connected 2023-08-17 16:56:56,889 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-17 16:56:56,890 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-17 16:56:56,890 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-17 16:56:56,897 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42619 2023-08-17 16:56:56,901 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42619 2023-08-17 16:56:56,901 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42619 2023-08-17 16:56:56,903 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42619 2023-08-17 16:56:56,905 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42619 2023-08-17 16:56:56,905 INFO [Listener at localhost/35639] master.HMaster(444): hbase.rootdir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85, hbase.cluster.distributed=false 2023-08-17 16:56:56,906 DEBUG [pool-560-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-08-17 16:56:56,917 INFO [Listener at localhost/35639] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-17 16:56:56,917 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:56,917 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:56,917 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:56:56,917 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:56:56,918 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:56:56,918 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:56:56,918 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37377 2023-08-17 16:56:56,919 INFO [Listener at localhost/35639] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-17 16:56:56,920 DEBUG [Listener at localhost/35639] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-17 16:56:56,921 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:56,922 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:56,924 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37377 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:56,926 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:373770x0, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:56,928 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:373770x0, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-17 16:56:56,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37377-0x1001b2c4f41002b connected 2023-08-17 16:56:56,929 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-17 16:56:56,930 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-17 16:56:56,931 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37377 2023-08-17 16:56:56,933 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37377 2023-08-17 16:56:56,933 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37377 2023-08-17 16:56:56,934 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37377 2023-08-17 16:56:56,934 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37377 2023-08-17 16:56:56,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:56:56,945 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-17 16:56:56,946 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:56:56,947 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-17 16:56:56,947 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-17 16:56:56,947 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:56,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-17 16:56:56,950 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-17 16:56:56,950 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,42619,1692291416875 from backup master directory 2023-08-17 16:56:56,952 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:56:56,952 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:56,952 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-17 16:56:56,952 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:56:56,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:56:56,991 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x68fe5b0e to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:56,995 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@168e29ec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:56,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-17 16:56:56,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-17 16:56:56,996 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:56,997 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:57,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,33151,1692291376904 to hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,33151,1692291376904-dead as it is dead 2023-08-17 16:56:57,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,33151,1692291376904-dead/jenkins-hbase4.apache.org%2C33151%2C1692291376904.jenkins-hbase4.apache.org%2C33151%2C1692291376904.regiongroup-0.1692291378504 2023-08-17 16:56:57,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,33151,1692291376904-dead/jenkins-hbase4.apache.org%2C33151%2C1692291376904.jenkins-hbase4.apache.org%2C33151%2C1692291376904.regiongroup-0.1692291378504 after 0ms 2023-08-17 16:56:57,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,33151,1692291376904-dead/jenkins-hbase4.apache.org%2C33151%2C1692291376904.jenkins-hbase4.apache.org%2C33151%2C1692291376904.regiongroup-0.1692291378504 to hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33151%2C1692291376904.jenkins-hbase4.apache.org%2C33151%2C1692291376904.regiongroup-0.1692291378504 2023-08-17 16:56:57,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,33151,1692291376904-dead 2023-08-17 16:56:57,005 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:56:57,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:57,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42619%2C1692291416875.jenkins-hbase4.apache.org%2C42619%2C1692291416875.regiongroup-0, suffix=, logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,42619,1692291416875, archiveDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/oldWALs, maxLogs=10 2023-08-17 16:56:57,023 DEBUG [RS-EventLoopGroup-14-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK] 2023-08-17 16:56:57,023 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK] 2023-08-17 16:56:57,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/WALs/jenkins-hbase4.apache.org,42619,1692291416875/jenkins-hbase4.apache.org%2C42619%2C1692291416875.jenkins-hbase4.apache.org%2C42619%2C1692291416875.regiongroup-0.1692291417008 2023-08-17 16:56:57,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK], DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK]] 2023-08-17 16:56:57,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:57,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:57,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:57,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:57,030 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:57,031 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-17 16:56:57,032 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-17 16:56:57,037 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/9f129a1dda534bd88a49ad3831664095 2023-08-17 16:56:57,037 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:57,037 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-08-17 16:56:57,038 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33151%2C1692291376904.jenkins-hbase4.apache.org%2C33151%2C1692291376904.regiongroup-0.1692291378504 2023-08-17 16:56:57,042 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33151%2C1692291376904.jenkins-hbase4.apache.org%2C33151%2C1692291376904.regiongroup-0.1692291378504 2023-08-17 16:56:57,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33151%2C1692291376904.jenkins-hbase4.apache.org%2C33151%2C1692291376904.regiongroup-0.1692291378504 2023-08-17 16:56:57,046 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:56:57,048 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-08-17 16:56:57,049 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73759900, jitterRate=0.09910815954208374}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-17 16:56:57,049 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:56:57,049 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-17 16:56:57,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-17 16:56:57,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-17 16:56:57,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-17 16:56:57,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-17 16:56:57,055 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-17 16:56:57,055 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-17 16:56:57,056 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-17 16:56:57,056 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-17 16:56:57,056 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-08-17 16:56:57,056 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-08-17 16:56:57,057 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-08-17 16:56:57,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-17 16:56:57,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-17 16:56:57,059 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-17 16:56:57,059 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,45379,1692291378053, table=hbase:meta, region=1588230740 2023-08-17 16:56:57,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-08-17 16:56:57,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42469,1692291378095 already deleted, retry=false 2023-08-17 16:56:57,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,42469,1692291378095 on jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:56:57,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,42469,1692291378095, splitWal=true, meta=false 2023-08-17 16:56:57,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,42469,1692291378095 (carryingMeta=false) jenkins-hbase4.apache.org,42469,1692291378095/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@7575ef0c[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-17 16:56:57,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,45379,1692291378053 already deleted, retry=false 2023-08-17 16:56:57,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45379,1692291378053 on jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:56:57,065 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45379,1692291378053, splitWal=true, meta=true 2023-08-17 16:56:57,065 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase4.apache.org,45379,1692291378053 (carryingMeta=true) jenkins-hbase4.apache.org,45379,1692291378053/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@7f39d5dc[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-17 16:56:57,075 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-17 16:56:57,075 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-17 16:56:57,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-08-17 16:56:57,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-17 16:56:57,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-08-17 16:56:57,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-08-17 16:56:57,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-08-17 16:56:57,080 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-08-17 16:56:57,085 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-17 16:56:57,085 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-17 16:56:57,085 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:56:57,085 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,42619,1692291416875, sessionid=0x1001b2c4f41002a, setting cluster-up flag (Was=false) 2023-08-17 16:56:57,091 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-08-17 16:56:57,092 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:56:57,094 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-08-17 16:56:57,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:56:57,096 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/.hbase-snapshot/.tmp 2023-08-17 16:56:57,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:57,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:57,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:57,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:56:57,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-17 16:56:57,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:57,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692291447104 2023-08-17 16:56:57,105 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45379,1692291378053; numProcessing=1 2023-08-17 16:56:57,105 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-17 16:56:57,105 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45379,1692291378053, splitWal=true, meta=true 2023-08-17 16:56:57,105 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-17 16:56:57,105 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-17 16:56:57,105 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-17 16:56:57,105 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-17 16:56:57,105 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-17 16:56:57,105 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42469,1692291378095; numProcessing=2 2023-08-17 16:56:57,106 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42469,1692291378095, splitWal=true, meta=false 2023-08-17 16:56:57,106 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45379,1692291378053, splitWal=true, meta=true, isMeta: true 2023-08-17 16:56:57,109 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:57,110 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-17 16:56:57,110 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-17 16:56:57,110 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-17 16:56:57,111 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053-splitting 2023-08-17 16:56:57,112 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053-splitting dir is empty, no logs to split. 2023-08-17 16:56:57,112 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,45379,1692291378053 WAL count=0, meta=true 2023-08-17 16:56:57,113 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-17 16:56:57,113 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-17 16:56:57,114 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053-splitting dir is empty, no logs to split. 2023-08-17 16:56:57,114 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,45379,1692291378053 WAL count=0, meta=true 2023-08-17 16:56:57,114 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45379,1692291378053 WAL splitting is done? wals=0, meta=true 2023-08-17 16:56:57,115 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-17 16:56:57,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291417113,5,FailOnTimeoutGroup] 2023-08-17 16:56:57,117 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-17 16:56:57,120 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291417117,5,FailOnTimeoutGroup] 2023-08-17 16:56:57,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:57,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-17 16:56:57,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:57,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:57,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692291417120, completionTime=-1 2023-08-17 16:56:57,120 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-08-17 16:56:57,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-17 16:56:57,120 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:56:57,142 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(951): ClusterId : 1067016f-f973-4161-8f76-366ce33cec43 2023-08-17 16:56:57,142 DEBUG [RS:0;jenkins-hbase4:37377] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-17 16:56:57,144 DEBUG [RS:0;jenkins-hbase4:37377] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-17 16:56:57,144 DEBUG [RS:0;jenkins-hbase4:37377] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-17 16:56:57,146 DEBUG [RS:0;jenkins-hbase4:37377] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-17 16:56:57,148 DEBUG [RS:0;jenkins-hbase4:37377] zookeeper.ReadOnlyZKClient(139): Connect 0x2d27d836 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:57,152 DEBUG [RS:0;jenkins-hbase4:37377] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@23e023e2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:57,152 DEBUG [RS:0;jenkins-hbase4:37377] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27dfb346, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:56:57,165 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:37377 2023-08-17 16:56:57,165 INFO [RS:0;jenkins-hbase4:37377] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-17 16:56:57,165 INFO [RS:0;jenkins-hbase4:37377] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-17 16:56:57,165 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1022): About to register with Master. 2023-08-17 16:56:57,166 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,42619,1692291416875 with isa=jenkins-hbase4.apache.org/172.31.14.131:37377, startcode=1692291416917 2023-08-17 16:56:57,166 DEBUG [RS:0;jenkins-hbase4:37377] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-17 16:56:57,168 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50021, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-08-17 16:56:57,169 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=42619] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:56:57,169 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85 2023-08-17 16:56:57,170 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38511 2023-08-17 16:56:57,170 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-17 16:56:57,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-17 16:56:57,171 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-17 16:56:57,171 DEBUG [RS:0;jenkins-hbase4:37377] zookeeper.ZKUtil(162): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:56:57,172 WARN [RS:0;jenkins-hbase4:37377] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:56:57,172 INFO [RS:0;jenkins-hbase4:37377] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:57,173 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37377,1692291416917] 2023-08-17 16:56:57,173 INFO [RS:0;jenkins-hbase4:37377] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:57,173 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1948): logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:56:57,187 INFO [RS:0;jenkins-hbase4:37377] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-17 16:56:57,188 DEBUG [RS:0;jenkins-hbase4:37377] zookeeper.ZKUtil(162): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:56:57,189 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:56:57,189 INFO [RS:0;jenkins-hbase4:37377] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-17 16:56:57,190 INFO [RS:0;jenkins-hbase4:37377] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-17 16:56:57,191 INFO [RS:0;jenkins-hbase4:37377] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-17 16:56:57,191 INFO [RS:0;jenkins-hbase4:37377] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:57,191 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-17 16:56:57,192 INFO [RS:0;jenkins-hbase4:37377] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:57,193 DEBUG [RS:0;jenkins-hbase4:37377] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,193 DEBUG [RS:0;jenkins-hbase4:37377] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,193 DEBUG [RS:0;jenkins-hbase4:37377] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,193 DEBUG [RS:0;jenkins-hbase4:37377] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,193 DEBUG [RS:0;jenkins-hbase4:37377] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,193 DEBUG [RS:0;jenkins-hbase4:37377] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:56:57,193 DEBUG [RS:0;jenkins-hbase4:37377] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,193 DEBUG [RS:0;jenkins-hbase4:37377] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,193 DEBUG [RS:0;jenkins-hbase4:37377] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,193 DEBUG [RS:0;jenkins-hbase4:37377] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:56:57,195 INFO [RS:0;jenkins-hbase4:37377] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:57,195 INFO [RS:0;jenkins-hbase4:37377] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:57,195 INFO [RS:0;jenkins-hbase4:37377] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:57,206 INFO [RS:0;jenkins-hbase4:37377] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-17 16:56:57,206 INFO [RS:0;jenkins-hbase4:37377] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37377,1692291416917-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:56:57,217 INFO [RS:0;jenkins-hbase4:37377] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-17 16:56:57,218 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1692291396528.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase4.apache.org,37377,1692291416917] 2023-08-17 16:56:57,227 INFO [RS:0;jenkins-hbase4:37377] regionserver.Replication(203): jenkins-hbase4.apache.org,37377,1692291416917 started 2023-08-17 16:56:57,227 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,37377,1692291416917, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37377, sessionid=0x1001b2c4f41002b 2023-08-17 16:56:57,227 DEBUG [RS:0;jenkins-hbase4:37377] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-17 16:56:57,228 DEBUG [RS:0;jenkins-hbase4:37377] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:56:57,228 DEBUG [RS:0;jenkins-hbase4:37377] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37377,1692291416917' 2023-08-17 16:56:57,228 DEBUG [RS:0;jenkins-hbase4:37377] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-08-17 16:56:57,228 DEBUG [RS:0;jenkins-hbase4:37377] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-08-17 16:56:57,228 DEBUG [RS:0;jenkins-hbase4:37377] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-17 16:56:57,228 DEBUG [RS:0;jenkins-hbase4:37377] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-17 16:56:57,228 DEBUG [RS:0;jenkins-hbase4:37377] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:56:57,228 DEBUG [RS:0;jenkins-hbase4:37377] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37377,1692291416917' 2023-08-17 16:56:57,228 DEBUG [RS:0;jenkins-hbase4:37377] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-08-17 16:56:57,229 DEBUG [RS:0;jenkins-hbase4:37377] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-08-17 16:56:57,229 DEBUG [RS:0;jenkins-hbase4:37377] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-17 16:56:57,229 INFO [RS:0;jenkins-hbase4:37377] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-17 16:56:57,229 INFO [RS:0;jenkins-hbase4:37377] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-17 16:56:57,238 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x384a4232 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:56:57,241 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@637e20fb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:56:57,242 INFO [RS:0;jenkins-hbase4:37377.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:56:57,245 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:56:57,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1001b2c4f41002f connected 2023-08-17 16:56:57,248 INFO [RS:0;jenkins-hbase4:37377.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=1067016f-f973-4161-8f76-366ce33cec43 to cluster=54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:56:57,270 DEBUG [jenkins-hbase4:42619] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-17 16:56:57,271 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37377,1692291416917, state=OPENING 2023-08-17 16:56:57,273 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-08-17 16:56:57,273 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-17 16:56:57,273 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37377,1692291416917}] 2023-08-17 16:56:57,329 INFO [RS:0;jenkins-hbase4:37377] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:57,331 INFO [RS:0;jenkins-hbase4:37377] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0, suffix=, logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917, archiveDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs, maxLogs=10 2023-08-17 16:56:57,347 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK] 2023-08-17 16:56:57,347 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK] 2023-08-17 16:56:57,357 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0 for peer 1 2023-08-17 16:56:57,357 INFO [RS:0;jenkins-hbase4:37377] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0.1692291417331 2023-08-17 16:56:57,357 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0 2023-08-17 16:56:57,357 INFO [RS:0;jenkins-hbase4:37377] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:56:57,357 DEBUG [RS:0;jenkins-hbase4:37377] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK], DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK]] 2023-08-17 16:56:57,357 INFO [RS:0;jenkins-hbase4:37377.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0 2023-08-17 16:56:57,362 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:57,429 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:56:57,429 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:56:57,431 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51658, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-17 16:56:57,434 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-17 16:56:57,434 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:56:57,434 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:56:57,434 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:56:57,436 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37377%2C1692291416917.meta, suffix=.meta, logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917, archiveDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs, maxLogs=10 2023-08-17 16:56:57,448 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK] 2023-08-17 16:56:57,448 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK] 2023-08-17 16:56:57,450 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.meta.1692291417436.meta 2023-08-17 16:56:57,450 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK], DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK]] 2023-08-17 16:56:57,450 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:56:57,451 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-17 16:56:57,451 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-17 16:56:57,451 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-17 16:56:57,451 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-17 16:56:57,451 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:56:57,451 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-17 16:56:57,451 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-17 16:56:57,452 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-17 16:56:57,453 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/info 2023-08-17 16:56:57,453 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/info 2023-08-17 16:56:57,454 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-17 16:56:57,459 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/info/1084509cff0c4142899836b8c21573d3 2023-08-17 16:56:57,460 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:57,460 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-17 16:56:57,461 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:56:57,461 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:56:57,461 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-17 16:56:57,466 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:57,467 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/rep_barrier/9238b01861dd4e1cbed45fe870815754 2023-08-17 16:56:57,467 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:57,467 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-17 16:56:57,468 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/table 2023-08-17 16:56:57,468 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/table 2023-08-17 16:56:57,468 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-17 16:56:57,474 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/table/d480699fbb4445a09cbaf714b1a4e790 2023-08-17 16:56:57,474 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:56:57,475 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740 2023-08-17 16:56:57,476 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740 2023-08-17 16:56:57,478 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-17 16:56:57,479 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-17 16:56:57,480 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63476057, jitterRate=-0.05413304269313812}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-17 16:56:57,480 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-17 16:56:57,480 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1692291417429 2023-08-17 16:56:57,487 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-17 16:56:57,487 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-17 16:56:57,488 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37377,1692291416917, state=OPEN 2023-08-17 16:56:57,489 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-08-17 16:56:57,489 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-17 16:56:57,491 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-08-17 16:56:57,491 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37377,1692291416917 in 216 msec 2023-08-17 16:56:57,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-08-17 16:56:57,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 376 msec 2023-08-17 16:56:57,570 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:57,675 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:57,780 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:57,884 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:57,989 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:58,093 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:58,198 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:58,302 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:58,407 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:58,511 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:58,616 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:58,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1553ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-08-17 16:56:58,720 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:58,825 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:58,929 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:59,034 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:59,138 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:59,243 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:59,348 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:59,453 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:59,557 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:59,662 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:59,766 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:59,870 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:56:59,975 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:00,079 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:00,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-08-17 16:57:00,183 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:00,288 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:00,393 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:00,454 INFO [jenkins-hbase4:34993Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:57:00,455 INFO [jenkins-hbase4:34993Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-08-17 16:57:00,498 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:00,602 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:00,707 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:00,811 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:00,916 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,020 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,125 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,230 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,334 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,439 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,543 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-08-17 16:57:01,627 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-17 16:57:01,630 DEBUG [hconnection-0x340b059-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:57:01,631 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51660, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:57:01,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=e13cf09e05e627f75f15f5ddbd0b03a2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42469,1692291378095, regionLocation=jenkins-hbase4.apache.org,42469,1692291378095, openSeqNum=2 2023-08-17 16:57:01,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=4b04d98f902897a06d2d5c032dc77d72, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42469,1692291378095, regionLocation=jenkins-hbase4.apache.org,42469,1692291378095, openSeqNum=2 2023-08-17 16:57:01,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=a2668222e397f37ad3389946a7fe032f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45379,1692291378053, regionLocation=jenkins-hbase4.apache.org,45379,1692291378053, openSeqNum=2 2023-08-17 16:57:01,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-17 16:57:01,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692291481637 2023-08-17 16:57:01,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692291541638 2023-08-17 16:57:01,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 10 msec 2023-08-17 16:57:01,648 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,660 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45379,1692291378053 had 2 regions 2023-08-17 16:57:01,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42619,1692291416875-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:01,660 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,42469,1692291378095 had 2 regions 2023-08-17 16:57:01,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42619,1692291416875-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:01,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42619,1692291416875-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:01,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:42619, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:01,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:01,662 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. is NOT online; state={e13cf09e05e627f75f15f5ddbd0b03a2 state=OPEN, ts=1692291421637, server=jenkins-hbase4.apache.org,42469,1692291378095}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-08-17 16:57:01,663 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45379,1692291378053, splitWal=true, meta=true, isMeta: false 2023-08-17 16:57:01,663 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42469,1692291378095, splitWal=true, meta=false, isMeta: false 2023-08-17 16:57:01,666 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053-splitting dir is empty, no logs to split. 2023-08-17 16:57:01,666 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,45379,1692291378053 WAL count=0, meta=false 2023-08-17 16:57:01,666 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,42469,1692291378095/hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2., unknown_server=jenkins-hbase4.apache.org,42469,1692291378095/t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72., unknown_server=jenkins-hbase4.apache.org,45379,1692291378053/t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:57:01,666 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095-splitting 2023-08-17 16:57:01,667 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095-splitting dir is empty, no logs to split. 2023-08-17 16:57:01,667 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,42469,1692291378095 WAL count=0, meta=false 2023-08-17 16:57:01,668 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,45379,1692291378053-splitting dir is empty, no logs to split. 2023-08-17 16:57:01,668 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,45379,1692291378053 WAL count=0, meta=false 2023-08-17 16:57:01,668 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45379,1692291378053 WAL splitting is done? wals=0, meta=false 2023-08-17 16:57:01,669 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095-splitting dir is empty, no logs to split. 2023-08-17 16:57:01,669 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,42469,1692291378095 WAL count=0, meta=false 2023-08-17 16:57:01,669 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42469,1692291378095 WAL splitting is done? wals=0, meta=false 2023-08-17 16:57:01,670 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=a2668222e397f37ad3389946a7fe032f, ASSIGN}] 2023-08-17 16:57:01,671 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=a2668222e397f37ad3389946a7fe032f, ASSIGN 2023-08-17 16:57:01,671 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,42469,1692291378095 failed, ignore...File hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,42469,1692291378095-splitting does not exist. 2023-08-17 16:57:01,671 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=a2668222e397f37ad3389946a7fe032f, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:57:01,672 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e13cf09e05e627f75f15f5ddbd0b03a2, ASSIGN}, {pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=4b04d98f902897a06d2d5c032dc77d72, ASSIGN}] 2023-08-17 16:57:01,673 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e13cf09e05e627f75f15f5ddbd0b03a2, ASSIGN 2023-08-17 16:57:01,673 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=4b04d98f902897a06d2d5c032dc77d72, ASSIGN 2023-08-17 16:57:01,674 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=e13cf09e05e627f75f15f5ddbd0b03a2, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:57:01,674 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=4b04d98f902897a06d2d5c032dc77d72, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:57:01,674 DEBUG [jenkins-hbase4:42619] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-17 16:57:01,676 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=a2668222e397f37ad3389946a7fe032f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:01,676 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291421676"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291421676"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291421676"}]},"ts":"1692291421676"} 2023-08-17 16:57:01,676 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=e13cf09e05e627f75f15f5ddbd0b03a2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:01,677 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291421676"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291421676"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291421676"}]},"ts":"1692291421676"} 2023-08-17 16:57:01,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure a2668222e397f37ad3389946a7fe032f, server=jenkins-hbase4.apache.org,37377,1692291416917}] 2023-08-17 16:57:01,679 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure e13cf09e05e627f75f15f5ddbd0b03a2, server=jenkins-hbase4.apache.org,37377,1692291416917}] 2023-08-17 16:57:01,752 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,828 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=4b04d98f902897a06d2d5c032dc77d72, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:01,828 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291421828"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291421828"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291421828"}]},"ts":"1692291421828"} 2023-08-17 16:57:01,830 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=24, state=RUNNABLE; OpenRegionProcedure 4b04d98f902897a06d2d5c032dc77d72, server=jenkins-hbase4.apache.org,37377,1692291416917}] 2023-08-17 16:57:01,834 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:57:01,834 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:57:01,836 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1, suffix=, logDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917, archiveDir=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs, maxLogs=10 2023-08-17 16:57:01,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:57:01,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a2668222e397f37ad3389946a7fe032f, NAME => 't2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:57:01,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup a2668222e397f37ad3389946a7fe032f 2023-08-17 16:57:01,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:57:01,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a2668222e397f37ad3389946a7fe032f 2023-08-17 16:57:01,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a2668222e397f37ad3389946a7fe032f 2023-08-17 16:57:01,838 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a2668222e397f37ad3389946a7fe032f 2023-08-17 16:57:01,839 DEBUG [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/cf1 2023-08-17 16:57:01,839 DEBUG [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/cf1 2023-08-17 16:57:01,840 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2668222e397f37ad3389946a7fe032f columnFamilyName cf1 2023-08-17 16:57:01,846 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7f11d4a19b91431ebde61712f73a7bd8 2023-08-17 16:57:01,846 DEBUG [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] regionserver.HStore(539): loaded hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/cf1/7f11d4a19b91431ebde61712f73a7bd8 2023-08-17 16:57:01,846 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] regionserver.HStore(310): Store=a2668222e397f37ad3389946a7fe032f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:01,847 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region a2668222e397f37ad3389946a7fe032f 2023-08-17 16:57:01,848 DEBUG [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/norep 2023-08-17 16:57:01,848 DEBUG [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/norep 2023-08-17 16:57:01,848 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2668222e397f37ad3389946a7fe032f columnFamilyName norep 2023-08-17 16:57:01,851 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK] 2023-08-17 16:57:01,852 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK] 2023-08-17 16:57:01,856 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4ba41047d6f94283b79849d37770ba50 2023-08-17 16:57:01,856 DEBUG [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] regionserver.HStore(539): loaded hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/norep/4ba41047d6f94283b79849d37770ba50 2023-08-17 16:57:01,859 INFO [StoreOpener-a2668222e397f37ad3389946a7fe032f-1] regionserver.HStore(310): Store=a2668222e397f37ad3389946a7fe032f/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:01,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f 2023-08-17 16:57:01,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f 2023-08-17 16:57:01,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-17 16:57:01,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a2668222e397f37ad3389946a7fe032f 2023-08-17 16:57:01,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a2668222e397f37ad3389946a7fe032f; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73239611, jitterRate=0.09135524928569794}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-17 16:57:01,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a2668222e397f37ad3389946a7fe032f: 2023-08-17 16:57:01,870 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1 for peer 1 2023-08-17 16:57:01,870 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1.1692291421836 2023-08-17 16:57:01,871 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1 2023-08-17 16:57:01,871 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:57:01,871 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f., pid=25, masterSystemTime=1692291421830 2023-08-17 16:57:01,873 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43377,DS-014fa655-a3a0-4df1-a42d-845ba3b78c73,DISK], DatanodeInfoWithStorage[127.0.0.1:39023,DS-a7a8b288-0f3d-43f7-a490-ab6a39dcee5d,DISK]] 2023-08-17 16:57:01,874 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1 2023-08-17 16:57:01,874 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e13cf09e05e627f75f15f5ddbd0b03a2, NAME => 'hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:57:01,875 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:57:01,875 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:57:01,875 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:57:01,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:57:01,875 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:57:01,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:57:01,875 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=a2668222e397f37ad3389946a7fe032f, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:01,876 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291421875"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291421875"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291421875"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291421875"}]},"ts":"1692291421875"} 2023-08-17 16:57:01,877 INFO [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:57:01,878 DEBUG [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/info 2023-08-17 16:57:01,878 DEBUG [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/info 2023-08-17 16:57:01,879 INFO [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e13cf09e05e627f75f15f5ddbd0b03a2 columnFamilyName info 2023-08-17 16:57:01,879 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-08-17 16:57:01,879 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure a2668222e397f37ad3389946a7fe032f, server=jenkins-hbase4.apache.org,37377,1692291416917 in 199 msec 2023-08-17 16:57:01,881 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=19 2023-08-17 16:57:01,881 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45379,1692291378053 after splitting done 2023-08-17 16:57:01,881 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=a2668222e397f37ad3389946a7fe032f, ASSIGN in 209 msec 2023-08-17 16:57:01,881 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45379,1692291378053 from processing; numProcessing=1 2023-08-17 16:57:01,882 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-17 16:57:01,882 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45379,1692291378053, splitWal=true, meta=true in 4.8170 sec 2023-08-17 16:57:01,886 DEBUG [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] regionserver.HStore(539): loaded hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/info/0eb76268e85e45b6914ae6bee766a07e 2023-08-17 16:57:01,887 INFO [StoreOpener-e13cf09e05e627f75f15f5ddbd0b03a2-1] regionserver.HStore(310): Store=e13cf09e05e627f75f15f5ddbd0b03a2/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:01,887 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:57:01,889 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:57:01,892 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:57:01,893 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e13cf09e05e627f75f15f5ddbd0b03a2; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64587443, jitterRate=-0.03757210075855255}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-17 16:57:01,893 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e13cf09e05e627f75f15f5ddbd0b03a2: 2023-08-17 16:57:01,894 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2., pid=26, masterSystemTime=1692291421830 2023-08-17 16:57:01,896 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:57:01,896 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:57:01,896 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=e13cf09e05e627f75f15f5ddbd0b03a2, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:01,896 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291421896"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291421896"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291421896"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291421896"}]},"ts":"1692291421896"} 2023-08-17 16:57:01,899 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-08-17 16:57:01,899 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure e13cf09e05e627f75f15f5ddbd0b03a2, server=jenkins-hbase4.apache.org,37377,1692291416917 in 219 msec 2023-08-17 16:57:01,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=e13cf09e05e627f75f15f5ddbd0b03a2, ASSIGN in 227 msec 2023-08-17 16:57:01,979 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:01,979 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:57:01,979 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:01,979 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0.1692291417331, lastWalPosition=454, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:57:01,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:57:01,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4b04d98f902897a06d2d5c032dc77d72, NAME => 't1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:57:01,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:57:01,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:57:01,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:57:01,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:57:01,986 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-17 16:57:01,986 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:57:01,986 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-17 16:57:01,986 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1.1692291421836, lastWalPosition=422, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:57:01,987 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:57:01,988 DEBUG [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/cf1 2023-08-17 16:57:01,988 DEBUG [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/cf1 2023-08-17 16:57:01,988 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b04d98f902897a06d2d5c032dc77d72 columnFamilyName cf1 2023-08-17 16:57:01,994 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1f1861a6a844c60805674304ef77081 2023-08-17 16:57:01,994 DEBUG [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] regionserver.HStore(539): loaded hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/cf1/c1f1861a6a844c60805674304ef77081 2023-08-17 16:57:01,994 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] regionserver.HStore(310): Store=4b04d98f902897a06d2d5c032dc77d72/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:01,995 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:57:01,996 DEBUG [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/norep 2023-08-17 16:57:01,996 DEBUG [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/norep 2023-08-17 16:57:01,996 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b04d98f902897a06d2d5c032dc77d72 columnFamilyName norep 2023-08-17 16:57:02,002 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17767cd3834547738b3f53cdd6bcb6bf 2023-08-17 16:57:02,002 DEBUG [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] regionserver.HStore(539): loaded hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/norep/17767cd3834547738b3f53cdd6bcb6bf 2023-08-17 16:57:02,002 INFO [StoreOpener-4b04d98f902897a06d2d5c032dc77d72-1] regionserver.HStore(310): Store=4b04d98f902897a06d2d5c032dc77d72/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:02,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:57:02,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:57:02,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-17 16:57:02,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:57:02,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4b04d98f902897a06d2d5c032dc77d72; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74965016, jitterRate=0.11706578731536865}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-17 16:57:02,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4b04d98f902897a06d2d5c032dc77d72: 2023-08-17 16:57:02,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72., pid=27, masterSystemTime=1692291421982 2023-08-17 16:57:02,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:57:02,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:57:02,011 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=4b04d98f902897a06d2d5c032dc77d72, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:02,012 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291422011"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291422011"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291422011"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291422011"}]},"ts":"1692291422011"} 2023-08-17 16:57:02,014 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=24 2023-08-17 16:57:02,014 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=24, state=SUCCESS; OpenRegionProcedure 4b04d98f902897a06d2d5c032dc77d72, server=jenkins-hbase4.apache.org,37377,1692291416917 in 183 msec 2023-08-17 16:57:02,016 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=18 2023-08-17 16:57:02,016 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=4b04d98f902897a06d2d5c032dc77d72, ASSIGN in 342 msec 2023-08-17 16:57:02,016 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,42469,1692291378095 after splitting done 2023-08-17 16:57:02,016 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,42469,1692291378095 from processing; numProcessing=0 2023-08-17 16:57:02,017 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,42469,1692291378095, splitWal=true, meta=false in 4.9540 sec 2023-08-17 16:57:02,083 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:02,090 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:02,090 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:57:02,090 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:02,090 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1.1692291421836, lastWalPosition=791, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:57:02,164 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-17 16:57:02,227 INFO [jenkins-hbase4:37377Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0]: currently replicating from: hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0.1692291417331 at position: 454 walGroup [jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1]: currently replicating from: hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1.1692291421836 at position: 791 2023-08-17 16:57:02,233 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:02,251 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:02,340 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:02,354 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:02,444 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:02,458 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:02,548 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:02,562 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:02,652 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:02,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-08-17 16:57:02,666 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:02,675 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-08-17 16:57:02,678 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-08-17 16:57:02,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.726sec 2023-08-17 16:57:02,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-17 16:57:02,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-17 16:57:02,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-17 16:57:02,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42619,1692291416875-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-17 16:57:02,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42619,1692291416875-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-17 16:57:02,681 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-17 16:57:02,750 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(139): Connect 0x1c47558d to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:57:02,754 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27464c26, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:57:02,756 DEBUG [hconnection-0x4d844808-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:57:02,756 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:02,758 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51664, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:57:02,763 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-08-17 16:57:02,763 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1c47558d to 127.0.0.1:56119 2023-08-17 16:57:02,763 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:02,764 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:42619 after: jenkins-hbase4.apache.org:42619 2023-08-17 16:57:02,764 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(139): Connect 0x12cce40a to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:57:02,768 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7de72a53, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:57:02,768 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x210e00dc to 127.0.0.1:56119 2023-08-17 16:57:02,768 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:02,768 DEBUG [Listener at localhost/35639] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-17 16:57:02,768 DEBUG [Listener at localhost/35639] util.JVMClusterUtil(257): Found active master hash=224363007, stopped=false 2023-08-17 16:57:02,768 INFO [Listener at localhost/35639] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:57:02,770 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:02,771 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-17 16:57:02,771 INFO [Listener at localhost/35639] procedure2.ProcedureExecutor(629): Stopping 2023-08-17 16:57:02,771 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-17 16:57:02,771 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:57:02,772 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x289436ae to 127.0.0.1:56119 2023-08-17 16:57:02,772 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:02,773 INFO [Listener at localhost/35639] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34993,1692291390151' ***** 2023-08-17 16:57:02,773 INFO [Listener at localhost/35639] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-17 16:57:02,773 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:57:02,773 INFO [RS:0;jenkins-hbase4:34993] regionserver.HeapMemoryManager(220): Stopping 2023-08-17 16:57:02,773 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:57:02,774 INFO [RS:0;jenkins-hbase4:34993] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-17 16:57:02,774 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-17 16:57:02,774 INFO [RS:0;jenkins-hbase4:34993] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-17 16:57:02,775 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(3305): Received CLOSE for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:57:02,775 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(3305): Received CLOSE for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:02,775 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(3305): Received CLOSE for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:57:02,775 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:57:02,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7b7c5dd31d075846c031f622b5c28bc1, disabling compactions & flushes 2023-08-17 16:57:02,776 DEBUG [RS:0;jenkins-hbase4:34993] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5724d046 to 127.0.0.1:56119 2023-08-17 16:57:02,776 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:02,776 DEBUG [RS:0;jenkins-hbase4:34993] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:02,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:02,776 INFO [RS:0;jenkins-hbase4:34993] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-17 16:57:02,776 INFO [RS:0;jenkins-hbase4:34993] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-17 16:57:02,776 INFO [RS:0;jenkins-hbase4:34993] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-17 16:57:02,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. after waiting 0 ms 2023-08-17 16:57:02,776 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-17 16:57:02,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:02,776 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 7b7c5dd31d075846c031f622b5c28bc1 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-08-17 16:57:02,776 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-17 16:57:02,776 DEBUG [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1478): Online Regions={7b7c5dd31d075846c031f622b5c28bc1=t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1., ec395d682a88bce1d4eae00d5f8e72c7=t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7., 1588230740=hbase:meta,,1.1588230740, 37b9df605a0d48d3a0fcac9d8ab4c0c4=hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.} 2023-08-17 16:57:02,777 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-17 16:57:02,777 DEBUG [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1504): Waiting on 1588230740, 37b9df605a0d48d3a0fcac9d8ab4c0c4, 7b7c5dd31d075846c031f622b5c28bc1, ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:02,777 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-17 16:57:02,777 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-17 16:57:02,777 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-17 16:57:02,777 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-17 16:57:02,777 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-08-17 16:57:02,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/.tmp/cf1/6fb92a896eb74831961608bba2eef14b 2023-08-17 16:57:02,789 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/.tmp/info/4dee513191c14004ab80c6f18c55997a 2023-08-17 16:57:02,794 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fb92a896eb74831961608bba2eef14b 2023-08-17 16:57:02,795 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/.tmp/info/4dee513191c14004ab80c6f18c55997a as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info/4dee513191c14004ab80c6f18c55997a 2023-08-17 16:57:02,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/.tmp/cf1/6fb92a896eb74831961608bba2eef14b as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1/6fb92a896eb74831961608bba2eef14b 2023-08-17 16:57:02,800 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info/4dee513191c14004ab80c6f18c55997a, entries=24, sequenceid=31, filesize=7.4 K 2023-08-17 16:57:02,800 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fb92a896eb74831961608bba2eef14b 2023-08-17 16:57:02,800 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1/6fb92a896eb74831961608bba2eef14b, entries=50, sequenceid=13, filesize=6.4 K 2023-08-17 16:57:02,801 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 24ms, sequenceid=31, compaction requested=false 2023-08-17 16:57:02,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for 7b7c5dd31d075846c031f622b5c28bc1 in 25ms, sequenceid=13, compaction requested=false 2023-08-17 16:57:02,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/recovered.edits/16.seqid, newMaxSeqId=16, maxSeqId=9 2023-08-17 16:57:02,817 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-08-17 16:57:02,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:02,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7b7c5dd31d075846c031f622b5c28bc1: 2023-08-17 16:57:02,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:02,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ec395d682a88bce1d4eae00d5f8e72c7, disabling compactions & flushes 2023-08-17 16:57:02,819 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:02,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:02,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. after waiting 0 ms 2023-08-17 16:57:02,819 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-17 16:57:02,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:02,820 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-17 16:57:02,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ec395d682a88bce1d4eae00d5f8e72c7 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-08-17 16:57:02,820 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-17 16:57:02,820 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-17 16:57:02,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=15 (bloomFilter=true), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/.tmp/cf1/0fcdfc43fa4d4bd8bcfb1fb4a19d2447 2023-08-17 16:57:02,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0fcdfc43fa4d4bd8bcfb1fb4a19d2447 2023-08-17 16:57:02,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/.tmp/cf1/0fcdfc43fa4d4bd8bcfb1fb4a19d2447 as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1/0fcdfc43fa4d4bd8bcfb1fb4a19d2447 2023-08-17 16:57:02,840 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0fcdfc43fa4d4bd8bcfb1fb4a19d2447 2023-08-17 16:57:02,840 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1/0fcdfc43fa4d4bd8bcfb1fb4a19d2447, entries=100, sequenceid=15, filesize=7.9 K 2023-08-17 16:57:02,841 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for ec395d682a88bce1d4eae00d5f8e72c7 in 22ms, sequenceid=15, compaction requested=false 2023-08-17 16:57:02,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/recovered.edits/18.seqid, newMaxSeqId=18, maxSeqId=11 2023-08-17 16:57:02,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:02,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ec395d682a88bce1d4eae00d5f8e72c7: 2023-08-17 16:57:02,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:02,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 37b9df605a0d48d3a0fcac9d8ab4c0c4, disabling compactions & flushes 2023-08-17 16:57:02,851 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:02,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:02,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. after waiting 0 ms 2023-08-17 16:57:02,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:02,853 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:57:02,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-08-17 16:57:02,855 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:02,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 37b9df605a0d48d3a0fcac9d8ab4c0c4: 2023-08-17 16:57:02,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:02,860 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:02,873 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:02,964 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:02,976 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:02,977 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34993,1692291390151; all regions closed. 2023-08-17 16:57:02,983 DEBUG [RS:0;jenkins-hbase4:34993] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:57:02,983 INFO [RS:0;jenkins-hbase4:34993] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34993%2C1692291390151.meta:.meta(num 1692291390712) 2023-08-17 16:57:02,993 DEBUG [RS:0;jenkins-hbase4:34993] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:57:02,993 INFO [RS:0;jenkins-hbase4:34993] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34993%2C1692291390151.jenkins-hbase4.apache.org%2C34993%2C1692291390151.regiongroup-0:(num 1692291390559) 2023-08-17 16:57:02,998 DEBUG [RS:0;jenkins-hbase4:34993] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:57:02,998 INFO [RS:0;jenkins-hbase4:34993] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34993%2C1692291390151.jenkins-hbase4.apache.org%2C34993%2C1692291390151.regiongroup-1:(num 1692291395115) 2023-08-17 16:57:02,998 DEBUG [RS:0;jenkins-hbase4:34993] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:02,998 INFO [RS:0;jenkins-hbase4:34993] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:57:02,998 INFO [RS:0;jenkins-hbase4:34993] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-17 16:57:02,998 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:57:02,999 DEBUG [RS:0;jenkins-hbase4:34993] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x44198932 to 127.0.0.1:56119 2023-08-17 16:57:02,999 DEBUG [RS:0;jenkins-hbase4:34993] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:02,999 INFO [RS:0;jenkins-hbase4:34993] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34993 2023-08-17 16:57:03,004 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410029, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:03,004 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,34993,1692291390151 2023-08-17 16:57:03,004 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:57:03,004 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:03,004 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:03,004 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410026, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:03,004 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:57:03,005 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34993,1692291390151] 2023-08-17 16:57:03,005 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34993,1692291390151; numProcessing=1 2023-08-17 16:57:03,007 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,34993,1692291390151 already deleted, retry=false 2023-08-17 16:57:03,007 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34993,1692291390151 expired; onlineServers=0 2023-08-17 16:57:03,007 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38743,1692291390124' ***** 2023-08-17 16:57:03,007 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-17 16:57:03,008 DEBUG [M:0;jenkins-hbase4:38743] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@960148c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:57:03,008 INFO [M:0;jenkins-hbase4:38743] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38743,1692291390124 2023-08-17 16:57:03,008 INFO [M:0;jenkins-hbase4:38743] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38743,1692291390124; all regions closed. 2023-08-17 16:57:03,008 DEBUG [M:0;jenkins-hbase4:38743] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:03,008 DEBUG [M:0;jenkins-hbase4:38743] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-17 16:57:03,008 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-17 16:57:03,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291390387] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291390387,5,FailOnTimeoutGroup] 2023-08-17 16:57:03,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291390387] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291390387,5,FailOnTimeoutGroup] 2023-08-17 16:57:03,008 DEBUG [M:0;jenkins-hbase4:38743] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-17 16:57:03,010 INFO [M:0;jenkins-hbase4:38743] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-17 16:57:03,010 INFO [M:0;jenkins-hbase4:38743] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-17 16:57:03,010 INFO [M:0;jenkins-hbase4:38743] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-17 16:57:03,010 DEBUG [M:0;jenkins-hbase4:38743] master.HMaster(1512): Stopping service threads 2023-08-17 16:57:03,010 INFO [M:0;jenkins-hbase4:38743] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-17 16:57:03,010 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-08-17 16:57:03,011 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:57:03,011 ERROR [M:0;jenkins-hbase4:38743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-17 16:57:03,011 INFO [M:0;jenkins-hbase4:38743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-17 16:57:03,011 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-17 16:57:03,011 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:57:03,011 DEBUG [M:0;jenkins-hbase4:38743] zookeeper.ZKUtil(398): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-08-17 16:57:03,011 WARN [M:0;jenkins-hbase4:38743] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-17 16:57:03,012 INFO [M:0;jenkins-hbase4:38743] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-17 16:57:03,012 INFO [M:0;jenkins-hbase4:38743] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-17 16:57:03,013 DEBUG [M:0;jenkins-hbase4:38743] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-17 16:57:03,013 INFO [M:0;jenkins-hbase4:38743] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:03,013 DEBUG [M:0;jenkins-hbase4:38743] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:03,013 DEBUG [M:0;jenkins-hbase4:38743] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-17 16:57:03,013 DEBUG [M:0;jenkins-hbase4:38743] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:03,013 INFO [M:0;jenkins-hbase4:38743] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=36.38 KB heapSize=46.77 KB 2023-08-17 16:57:03,027 INFO [M:0;jenkins-hbase4:38743] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=36.38 KB at sequenceid=238 (bloomFilter=true), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/62be486d835f4ddab498c7e81e8350b6 2023-08-17 16:57:03,032 INFO [M:0;jenkins-hbase4:38743] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 62be486d835f4ddab498c7e81e8350b6 2023-08-17 16:57:03,033 DEBUG [M:0;jenkins-hbase4:38743] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/62be486d835f4ddab498c7e81e8350b6 as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/62be486d835f4ddab498c7e81e8350b6 2023-08-17 16:57:03,038 INFO [M:0;jenkins-hbase4:38743] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 62be486d835f4ddab498c7e81e8350b6 2023-08-17 16:57:03,038 INFO [M:0;jenkins-hbase4:38743] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/62be486d835f4ddab498c7e81e8350b6, entries=20, sequenceid=238, filesize=5.6 K 2023-08-17 16:57:03,038 INFO [M:0;jenkins-hbase4:38743] regionserver.HRegion(2948): Finished flush of dataSize ~36.38 KB/37256, heapSize ~46.75 KB/47872, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 25ms, sequenceid=238, compaction requested=false 2023-08-17 16:57:03,040 INFO [M:0;jenkins-hbase4:38743] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:03,040 DEBUG [M:0;jenkins-hbase4:38743] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:57:03,044 INFO [M:0;jenkins-hbase4:38743] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-17 16:57:03,044 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:57:03,044 INFO [M:0;jenkins-hbase4:38743] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38743 2023-08-17 16:57:03,047 DEBUG [M:0;jenkins-hbase4:38743] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,38743,1692291390124 already deleted, retry=false 2023-08-17 16:57:03,068 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:03,080 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:03,172 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:03,183 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:03,267 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:03,267 INFO [M:0;jenkins-hbase4:38743] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38743,1692291390124; zookeeper connection closed. 2023-08-17 16:57:03,267 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:38743-0x1001b2c4f41001c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:03,276 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:03,287 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-08-17 16:57:03,367 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:03,367 INFO [RS:0;jenkins-hbase4:34993] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34993,1692291390151; zookeeper connection closed. 2023-08-17 16:57:03,367 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:34993-0x1001b2c4f41001d, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:03,369 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@677d675c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@677d675c 2023-08-17 16:57:03,369 INFO [Listener at localhost/35639] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-17 16:57:03,371 DEBUG [hconnection-0x47074a73-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:57:03,373 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51666, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:57:03,381 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-08-17 16:57:03,392 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1926 2023-08-17 16:57:03,392 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 11 WAL entries eligible for replication 2023-08-17 16:57:03,392 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1926 2023-08-17 16:57:03,392 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4b04d98f902897a06d2d5c032dc77d72/112=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/113=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/114=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/115=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/116=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/117=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/118=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/119=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/120=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/121=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/122=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1.1692291421836, lastWalPosition=1926, nbRowKeys=11, nbHFiles=0, heapSize=2288, lastSeqIds={}, endOfFile=false,usedBufferSize=2288] 2023-08-17 16:57:03,395 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-17 16:57:03,487 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 2130 2023-08-17 16:57:03,487 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 16 WAL entries eligible for replication 2023-08-17 16:57:03,487 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 2130 2023-08-17 16:57:03,487 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/a2668222e397f37ad3389946a7fe032f/212=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/213=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/214=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/215=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/216=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/217=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/218=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/219=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/220=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/221=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/222=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/223=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/224=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/225=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/226=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/227=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0.1692291417331, lastWalPosition=2130, nbRowKeys=16, nbHFiles=0, heapSize=3328, lastSeqIds={}, endOfFile=false,usedBufferSize=3328] 2023-08-17 16:57:03,489 WARN [RS:0;jenkins-hbase4:37377.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-17 16:57:03,495 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-17 16:57:03,501 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 11467 2023-08-17 16:57:03,501 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 89 WAL entries eligible for replication 2023-08-17 16:57:03,501 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 11467 2023-08-17 16:57:03,590 WARN [RS:0;jenkins-hbase4:37377.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-17 16:57:03,605 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 16542 2023-08-17 16:57:03,605 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 135 WAL entries eligible for replication 2023-08-17 16:57:03,605 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 16542 2023-08-17 16:57:03,606 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 11467 2023-08-17 16:57:03,653 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x12cce40a to 127.0.0.1:56119 2023-08-17 16:57:03,653 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:03,654 DEBUG [Listener at localhost/35639] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-17 16:57:03,654 DEBUG [Listener at localhost/35639] util.JVMClusterUtil(257): Found active master hash=218018092, stopped=false 2023-08-17 16:57:03,654 INFO [Listener at localhost/35639] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:57:03,656 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-17 16:57:03,656 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-17 16:57:03,656 INFO [Listener at localhost/35639] procedure2.ProcedureExecutor(629): Stopping 2023-08-17 16:57:03,657 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:57:03,659 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x68fe5b0e to 127.0.0.1:56119 2023-08-17 16:57:03,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-17 16:57:03,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-17 16:57:03,659 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:03,660 INFO [Listener at localhost/35639] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37377,1692291416917' ***** 2023-08-17 16:57:03,660 INFO [Listener at localhost/35639] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-17 16:57:03,660 INFO [RS:0;jenkins-hbase4:37377] regionserver.HeapMemoryManager(220): Stopping 2023-08-17 16:57:03,660 INFO [RS:0;jenkins-hbase4:37377] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-17 16:57:03,660 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-17 16:57:03,660 INFO [RS:0;jenkins-hbase4:37377] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-17 16:57:03,661 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(3305): Received CLOSE for a2668222e397f37ad3389946a7fe032f 2023-08-17 16:57:03,661 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(3305): Received CLOSE for e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:57:03,661 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(3305): Received CLOSE for 4b04d98f902897a06d2d5c032dc77d72 2023-08-17 16:57:03,661 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:03,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a2668222e397f37ad3389946a7fe032f, disabling compactions & flushes 2023-08-17 16:57:03,662 DEBUG [RS:0;jenkins-hbase4:37377] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2d27d836 to 127.0.0.1:56119 2023-08-17 16:57:03,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:57:03,662 DEBUG [RS:0;jenkins-hbase4:37377] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:03,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:57:03,662 INFO [RS:0;jenkins-hbase4:37377] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-17 16:57:03,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. after waiting 0 ms 2023-08-17 16:57:03,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:57:03,662 INFO [RS:0;jenkins-hbase4:37377] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-17 16:57:03,662 INFO [RS:0;jenkins-hbase4:37377] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-17 16:57:03,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing a2668222e397f37ad3389946a7fe032f 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-08-17 16:57:03,662 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-17 16:57:03,663 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-17 16:57:03,663 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1478): Online Regions={a2668222e397f37ad3389946a7fe032f=t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f., 1588230740=hbase:meta,,1.1588230740, e13cf09e05e627f75f15f5ddbd0b03a2=hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2., 4b04d98f902897a06d2d5c032dc77d72=t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72.} 2023-08-17 16:57:03,663 DEBUG [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1504): Waiting on 1588230740, 4b04d98f902897a06d2d5c032dc77d72, a2668222e397f37ad3389946a7fe032f, e13cf09e05e627f75f15f5ddbd0b03a2 2023-08-17 16:57:03,663 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-17 16:57:03,663 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-17 16:57:03,663 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-17 16:57:03,663 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-17 16:57:03,663 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-17 16:57:03,664 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-08-17 16:57:03,677 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/.tmp/info/e4ee310b0f1e4805b2974a236619db30 2023-08-17 16:57:03,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/.tmp/cf1/c3b751239a2548f4a4263d62d192d0bc 2023-08-17 16:57:03,690 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/.tmp/rep_barrier/01e3142fed364927a7c7e1a19ffbbd77 2023-08-17 16:57:03,695 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/.tmp/norep/54518d9ae9cf4b33be8c090b2e82cb8b 2023-08-17 16:57:03,696 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/.tmp/info/e4ee310b0f1e4805b2974a236619db30 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/info/e4ee310b0f1e4805b2974a236619db30 2023-08-17 16:57:03,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/.tmp/cf1/c3b751239a2548f4a4263d62d192d0bc as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/cf1/c3b751239a2548f4a4263d62d192d0bc 2023-08-17 16:57:03,701 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/info/e4ee310b0f1e4805b2974a236619db30, entries=24, sequenceid=31, filesize=7.4 K 2023-08-17 16:57:03,702 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/.tmp/rep_barrier/01e3142fed364927a7c7e1a19ffbbd77 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/rep_barrier/01e3142fed364927a7c7e1a19ffbbd77 2023-08-17 16:57:03,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/cf1/c3b751239a2548f4a4263d62d192d0bc, entries=200, sequenceid=413, filesize=12.5 K 2023-08-17 16:57:03,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/.tmp/norep/54518d9ae9cf4b33be8c090b2e82cb8b as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/norep/54518d9ae9cf4b33be8c090b2e82cb8b 2023-08-17 16:57:03,709 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/rep_barrier/01e3142fed364927a7c7e1a19ffbbd77, entries=2, sequenceid=31, filesize=4.9 K 2023-08-17 16:57:03,711 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 11467 2023-08-17 16:57:03,712 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 48ms, sequenceid=31, compaction requested=false 2023-08-17 16:57:03,713 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/norep/54518d9ae9cf4b33be8c090b2e82cb8b, entries=1, sequenceid=413, filesize=4.8 K 2023-08-17 16:57:03,714 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for a2668222e397f37ad3389946a7fe032f in 52ms, sequenceid=413, compaction requested=false 2023-08-17 16:57:03,714 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 22189 2023-08-17 16:57:03,714 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 49 WAL entries eligible for replication 2023-08-17 16:57:03,719 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:57:03,733 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t2_syncup/a2668222e397f37ad3389946a7fe032f/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-08-17 16:57:03,733 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-08-17 16:57:03,734 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:57:03,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a2668222e397f37ad3389946a7fe032f: 2023-08-17 16:57:03,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1692291383743.a2668222e397f37ad3389946a7fe032f. 2023-08-17 16:57:03,735 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-17 16:57:03,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e13cf09e05e627f75f15f5ddbd0b03a2, disabling compactions & flushes 2023-08-17 16:57:03,735 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-17 16:57:03,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:57:03,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:57:03,735 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-17 16:57:03,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. after waiting 0 ms 2023-08-17 16:57:03,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:57:03,736 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-17 16:57:03,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/hbase/namespace/e13cf09e05e627f75f15f5ddbd0b03a2/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-08-17 16:57:03,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:57:03,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e13cf09e05e627f75f15f5ddbd0b03a2: 2023-08-17 16:57:03,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692291379881.e13cf09e05e627f75f15f5ddbd0b03a2. 2023-08-17 16:57:03,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4b04d98f902897a06d2d5c032dc77d72, disabling compactions & flushes 2023-08-17 16:57:03,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:57:03,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:57:03,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. after waiting 0 ms 2023-08-17 16:57:03,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:57:03,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 4b04d98f902897a06d2d5c032dc77d72 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-08-17 16:57:03,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/.tmp/cf1/4e5f44d805d14f2b8cb8c165776a3736 2023-08-17 16:57:03,763 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-17 16:57:03,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/.tmp/norep/7d512ef9c4c942d39c7210d083128ddb 2023-08-17 16:57:03,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/.tmp/cf1/4e5f44d805d14f2b8cb8c165776a3736 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/cf1/4e5f44d805d14f2b8cb8c165776a3736 2023-08-17 16:57:03,781 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/cf1/4e5f44d805d14f2b8cb8c165776a3736, entries=100, sequenceid=213, filesize=8.4 K 2023-08-17 16:57:03,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/.tmp/norep/7d512ef9c4c942d39c7210d083128ddb as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/norep/7d512ef9c4c942d39c7210d083128ddb 2023-08-17 16:57:03,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/norep/7d512ef9c4c942d39c7210d083128ddb, entries=1, sequenceid=213, filesize=4.8 K 2023-08-17 16:57:03,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for 4b04d98f902897a06d2d5c032dc77d72 in 47ms, sequenceid=213, compaction requested=false 2023-08-17 16:57:03,792 WARN [RS:0;jenkins-hbase4:37377.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-17 16:57:03,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/data/default/t1_syncup/4b04d98f902897a06d2d5c032dc77d72/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-08-17 16:57:03,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:57:03,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4b04d98f902897a06d2d5c032dc77d72: 2023-08-17 16:57:03,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1692291383106.4b04d98f902897a06d2d5c032dc77d72. 2023-08-17 16:57:03,815 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 12631 2023-08-17 16:57:03,815 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-17 16:57:03,863 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37377,1692291416917; all regions closed. 2023-08-17 16:57:03,871 DEBUG [RS:0;jenkins-hbase4:37377] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs 2023-08-17 16:57:03,871 INFO [RS:0;jenkins-hbase4:37377] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37377%2C1692291416917.meta:.meta(num 1692291417436) 2023-08-17 16:57:03,876 DEBUG [RS:0;jenkins-hbase4:37377] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs 2023-08-17 16:57:03,876 INFO [RS:0;jenkins-hbase4:37377] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1:(num 1692291421836) 2023-08-17 16:57:03,879 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/WALs/jenkins-hbase4.apache.org,37377,1692291416917/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0.1692291417331 not finished, retry = 0 2023-08-17 16:57:03,981 DEBUG [RS:0;jenkins-hbase4:37377] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs 2023-08-17 16:57:03,981 INFO [RS:0;jenkins-hbase4:37377] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0:(num 1692291417331) 2023-08-17 16:57:03,981 DEBUG [RS:0;jenkins-hbase4:37377] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:03,981 INFO [RS:0;jenkins-hbase4:37377] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:57:03,982 INFO [RS:0;jenkins-hbase4:37377] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-17 16:57:03,982 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:57:03,982 INFO [RS:0;jenkins-hbase4:37377] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-17 16:57:04,063 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-08-17 16:57:04,084 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f41002f, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:04,085 DEBUG [RS:0;jenkins-hbase4:37377] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x384a4232 to 127.0.0.1:56119 2023-08-17 16:57:04,085 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f41002f, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:04,085 DEBUG [RS:0;jenkins-hbase4:37377] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:04,187 DEBUG [RS:0;jenkins-hbase4:37377.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1] regionserver.HBaseInterClusterReplicationEndpoint(243): No sinks available at peer [Source for peer 1]: Interrupted while sleeping between retries 2023-08-17 16:57:04,286 INFO [RS:0;jenkins-hbase4:37377] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase4:37377.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1 terminated 2023-08-17 16:57:04,286 INFO [RS:0;jenkins-hbase4:37377] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1 terminated 2023-08-17 16:57:04,288 INFO [RS:0;jenkins-hbase4:37377] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37377 2023-08-17 16:57:04,292 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:04,292 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-17 16:57:04,292 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-17 16:57:04,294 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37377,1692291416917] 2023-08-17 16:57:04,294 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37377,1692291416917; numProcessing=1 2023-08-17 16:57:04,295 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,37377,1692291416917 already deleted, retry=false 2023-08-17 16:57:04,295 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37377,1692291416917 expired; onlineServers=0 2023-08-17 16:57:04,295 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42619,1692291416875' ***** 2023-08-17 16:57:04,295 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-17 16:57:04,295 DEBUG [M:0;jenkins-hbase4:42619] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@8c1246b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:57:04,296 INFO [M:0;jenkins-hbase4:42619] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42619,1692291416875 2023-08-17 16:57:04,296 INFO [M:0;jenkins-hbase4:42619] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42619,1692291416875; all regions closed. 2023-08-17 16:57:04,296 DEBUG [M:0;jenkins-hbase4:42619] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:04,296 DEBUG [M:0;jenkins-hbase4:42619] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-17 16:57:04,296 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-17 16:57:04,296 DEBUG [M:0;jenkins-hbase4:42619] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-17 16:57:04,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291417117] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291417117,5,FailOnTimeoutGroup] 2023-08-17 16:57:04,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291417113] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291417113,5,FailOnTimeoutGroup] 2023-08-17 16:57:04,296 INFO [M:0;jenkins-hbase4:42619] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-17 16:57:04,297 INFO [M:0;jenkins-hbase4:42619] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-17 16:57:04,297 INFO [M:0;jenkins-hbase4:42619] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-17 16:57:04,298 DEBUG [M:0;jenkins-hbase4:42619] master.HMaster(1512): Stopping service threads 2023-08-17 16:57:04,298 INFO [M:0;jenkins-hbase4:42619] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-17 16:57:04,298 ERROR [M:0;jenkins-hbase4:42619] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-17 16:57:04,298 INFO [M:0;jenkins-hbase4:42619] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-17 16:57:04,298 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-08-17 16:57:04,298 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-17 16:57:04,298 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-17 16:57:04,299 DEBUG [M:0;jenkins-hbase4:42619] zookeeper.ZKUtil(398): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-08-17 16:57:04,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-17 16:57:04,299 WARN [M:0;jenkins-hbase4:42619] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-17 16:57:04,299 INFO [M:0;jenkins-hbase4:42619] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-17 16:57:04,299 INFO [M:0;jenkins-hbase4:42619] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-17 16:57:04,301 DEBUG [M:0;jenkins-hbase4:42619] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-17 16:57:04,301 INFO [M:0;jenkins-hbase4:42619] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:04,301 DEBUG [M:0;jenkins-hbase4:42619] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:04,301 DEBUG [M:0;jenkins-hbase4:42619] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-17 16:57:04,301 DEBUG [M:0;jenkins-hbase4:42619] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:04,301 INFO [M:0;jenkins-hbase4:42619] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.71 KB heapSize=36.96 KB 2023-08-17 16:57:04,311 INFO [M:0;jenkins-hbase4:42619] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.71 KB at sequenceid=218 (bloomFilter=true), to=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/fb8d85e5b22645ccb3441935b2f81640 2023-08-17 16:57:04,316 DEBUG [M:0;jenkins-hbase4:42619] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/fb8d85e5b22645ccb3441935b2f81640 as hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/fb8d85e5b22645ccb3441935b2f81640 2023-08-17 16:57:04,320 INFO [M:0;jenkins-hbase4:42619] regionserver.HStore(1080): Added hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/fb8d85e5b22645ccb3441935b2f81640, entries=10, sequenceid=218, filesize=5.8 K 2023-08-17 16:57:04,321 INFO [M:0;jenkins-hbase4:42619] regionserver.HRegion(2948): Finished flush of dataSize ~29.71 KB/30420, heapSize ~36.95 KB/37832, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=218, compaction requested=false 2023-08-17 16:57:04,323 INFO [M:0;jenkins-hbase4:42619] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:04,323 DEBUG [M:0;jenkins-hbase4:42619] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:57:04,327 INFO [M:0;jenkins-hbase4:42619] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-17 16:57:04,327 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:57:04,328 INFO [M:0;jenkins-hbase4:42619] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42619 2023-08-17 16:57:04,331 DEBUG [M:0;jenkins-hbase4:42619] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,42619,1692291416875 already deleted, retry=false 2023-08-17 16:57:04,394 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:04,394 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:37377-0x1001b2c4f41002b, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:04,394 INFO [RS:0;jenkins-hbase4:37377] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37377,1692291416917; zookeeper connection closed. 2023-08-17 16:57:04,395 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@596cf2e3] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@596cf2e3 2023-08-17 16:57:04,396 INFO [Listener at localhost/35639] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-17 16:57:04,494 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:04,494 INFO [M:0;jenkins-hbase4:42619] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42619,1692291416875; zookeeper connection closed. 2023-08-17 16:57:04,494 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:42619-0x1001b2c4f41002a, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:04,497 INFO [Listener at localhost/35639] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-08-17 16:57:04,498 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:57:04,498 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:57:04,498 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:57:04,498 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:57:04,498 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:57:04,498 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:57:04,499 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34623 2023-08-17 16:57:04,499 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:57:04,500 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:57:04,501 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34623 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:57:04,503 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:346230x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:57:04,504 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:34623-0x1001b2c4f410032 connected 2023-08-17 16:57:04,507 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:57:04,507 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:57:04,508 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-17 16:57:04,508 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34623 2023-08-17 16:57:04,508 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34623 2023-08-17 16:57:04,511 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34623 2023-08-17 16:57:04,513 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34623 2023-08-17 16:57:04,517 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34623 2023-08-17 16:57:04,517 INFO [Listener at localhost/35639] master.HMaster(444): hbase.rootdir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4, hbase.cluster.distributed=false 2023-08-17 16:57:04,518 DEBUG [pool-662-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-08-17 16:57:04,529 INFO [Listener at localhost/35639] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-17 16:57:04,530 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:57:04,530 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-17 16:57:04,530 INFO [Listener at localhost/35639] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-17 16:57:04,530 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-17 16:57:04,530 INFO [Listener at localhost/35639] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-17 16:57:04,530 INFO [Listener at localhost/35639] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-17 16:57:04,531 INFO [Listener at localhost/35639] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40293 2023-08-17 16:57:04,531 INFO [Listener at localhost/35639] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-17 16:57:04,532 DEBUG [Listener at localhost/35639] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-17 16:57:04,533 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:57:04,534 INFO [Listener at localhost/35639] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:57:04,534 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40293 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:57:04,538 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:402930x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:57:04,540 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40293-0x1001b2c4f410033 connected 2023-08-17 16:57:04,540 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:57:04,540 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:57:04,541 DEBUG [Listener at localhost/35639] zookeeper.ZKUtil(164): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-17 16:57:04,541 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40293 2023-08-17 16:57:04,541 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40293 2023-08-17 16:57:04,541 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40293 2023-08-17 16:57:04,542 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40293 2023-08-17 16:57:04,542 DEBUG [Listener at localhost/35639] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40293 2023-08-17 16:57:04,543 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,34623,1692291424497 2023-08-17 16:57:04,544 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-17 16:57:04,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,34623,1692291424497 2023-08-17 16:57:04,546 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-17 16:57:04,546 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-17 16:57:04,546 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:57:04,547 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-17 16:57:04,548 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,34623,1692291424497 from backup master directory 2023-08-17 16:57:04,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-17 16:57:04,550 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,34623,1692291424497 2023-08-17 16:57:04,550 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:57:04,550 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,34623,1692291424497 2023-08-17 16:57:04,550 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-17 16:57:04,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-17 16:57:04,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x6aa846b2 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:57:04,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c00a30f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:57:04,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-17 16:57:04,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-17 16:57:04,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:57:04,588 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:57:04,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,38743,1692291390124 to hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,38743,1692291390124-dead as it is dead 2023-08-17 16:57:04,593 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,38743,1692291390124-dead/jenkins-hbase4.apache.org%2C38743%2C1692291390124.jenkins-hbase4.apache.org%2C38743%2C1692291390124.regiongroup-0.1692291390236 2023-08-17 16:57:04,593 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,38743,1692291390124-dead/jenkins-hbase4.apache.org%2C38743%2C1692291390124.jenkins-hbase4.apache.org%2C38743%2C1692291390124.regiongroup-0.1692291390236 after 0ms 2023-08-17 16:57:04,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,38743,1692291390124-dead/jenkins-hbase4.apache.org%2C38743%2C1692291390124.jenkins-hbase4.apache.org%2C38743%2C1692291390124.regiongroup-0.1692291390236 to hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38743%2C1692291390124.jenkins-hbase4.apache.org%2C38743%2C1692291390124.regiongroup-0.1692291390236 2023-08-17 16:57:04,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,38743,1692291390124-dead 2023-08-17 16:57:04,594 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34623,1692291424497 2023-08-17 16:57:04,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:57:04,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34623%2C1692291424497.jenkins-hbase4.apache.org%2C34623%2C1692291424497.regiongroup-0, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34623,1692291424497, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/oldWALs, maxLogs=10 2023-08-17 16:57:04,611 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK] 2023-08-17 16:57:04,612 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:57:04,612 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:57:04,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/WALs/jenkins-hbase4.apache.org,34623,1692291424497/jenkins-hbase4.apache.org%2C34623%2C1692291424497.jenkins-hbase4.apache.org%2C34623%2C1692291424497.regiongroup-0.1692291424596 2023-08-17 16:57:04,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK], DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK]] 2023-08-17 16:57:04,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:57:04,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:57:04,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:57:04,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:57:04,615 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:57:04,616 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-17 16:57:04,617 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-17 16:57:04,622 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/0c1abd12533b4b5aae99c934fcfdd5b9 2023-08-17 16:57:04,625 INFO [StoreFileOpener-proc-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 62be486d835f4ddab498c7e81e8350b6 2023-08-17 16:57:04,625 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/62be486d835f4ddab498c7e81e8350b6 2023-08-17 16:57:04,626 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:04,626 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-08-17 16:57:04,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38743%2C1692291390124.jenkins-hbase4.apache.org%2C38743%2C1692291390124.regiongroup-0.1692291390236 2023-08-17 16:57:04,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 139, firstSequenceIdInLog=120, maxSequenceIdInLog=240, path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38743%2C1692291390124.jenkins-hbase4.apache.org%2C38743%2C1692291390124.regiongroup-0.1692291390236 2023-08-17 16:57:04,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38743%2C1692291390124.jenkins-hbase4.apache.org%2C38743%2C1692291390124.regiongroup-0.1692291390236 2023-08-17 16:57:04,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-17 16:57:04,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/240.seqid, newMaxSeqId=240, maxSeqId=118 2023-08-17 16:57:04,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=241; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69352751, jitterRate=0.033436521887779236}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-17 16:57:04,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:57:04,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-17 16:57:04,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-17 16:57:04,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-17 16:57:04,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-17 16:57:04,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-17 16:57:04,644 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-17 16:57:04,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-17 16:57:04,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-17 16:57:04,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-17 16:57:04,646 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-08-17 16:57:04,646 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-08-17 16:57:04,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-17 16:57:04,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-17 16:57:04,647 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-17 16:57:04,647 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34993,1692291390151, table=hbase:meta, region=1588230740 2023-08-17 16:57:04,649 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-08-17 16:57:04,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,34993,1692291390151 already deleted, retry=false 2023-08-17 16:57:04,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34993,1692291390151 on jenkins-hbase4.apache.org,34623,1692291424497 2023-08-17 16:57:04,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34993,1692291390151, splitWal=true, meta=true 2023-08-17 16:57:04,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase4.apache.org,34993,1692291390151 (carryingMeta=true) jenkins-hbase4.apache.org,34993,1692291390151/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@9bd2ef6[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-17 16:57:04,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-17 16:57:04,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-17 16:57:04,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-08-17 16:57:04,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-17 16:57:04,663 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-08-17 16:57:04,664 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-08-17 16:57:04,664 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-08-17 16:57:04,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-08-17 16:57:04,667 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-17 16:57:04,667 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-17 16:57:04,667 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:57:04,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,34623,1692291424497, sessionid=0x1001b2c4f410032, setting cluster-up flag (Was=false) 2023-08-17 16:57:04,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-08-17 16:57:04,673 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34623,1692291424497 2023-08-17 16:57:04,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-08-17 16:57:04,677 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34623,1692291424497 2023-08-17 16:57:04,677 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/.hbase-snapshot/.tmp 2023-08-17 16:57:04,678 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:57:04,678 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:57:04,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:57:04,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-17 16:57:04,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-17 16:57:04,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:57:04,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692291454685 2023-08-17 16:57:04,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-17 16:57:04,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-17 16:57:04,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-17 16:57:04,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-17 16:57:04,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-17 16:57:04,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-17 16:57:04,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:04,687 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34993,1692291390151; numProcessing=1 2023-08-17 16:57:04,688 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-17 16:57:04,688 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34993,1692291390151, splitWal=true, meta=true 2023-08-17 16:57:04,688 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-17 16:57:04,688 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-17 16:57:04,688 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-17 16:57:04,688 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-17 16:57:04,688 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34993,1692291390151, splitWal=true, meta=true, isMeta: true 2023-08-17 16:57:04,692 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291424688,5,FailOnTimeoutGroup] 2023-08-17 16:57:04,692 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291424692,5,FailOnTimeoutGroup] 2023-08-17 16:57:04,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:04,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-17 16:57:04,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:04,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:04,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692291424692, completionTime=-1 2023-08-17 16:57:04,692 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-08-17 16:57:04,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-17 16:57:04,693 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151-splitting 2023-08-17 16:57:04,694 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151-splitting dir is empty, no logs to split. 2023-08-17 16:57:04,694 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34993,1692291390151 WAL count=0, meta=true 2023-08-17 16:57:04,696 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151-splitting dir is empty, no logs to split. 2023-08-17 16:57:04,696 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34993,1692291390151 WAL count=0, meta=true 2023-08-17 16:57:04,696 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34993,1692291390151 WAL splitting is done? wals=0, meta=true 2023-08-17 16:57:04,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-17 16:57:04,698 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-17 16:57:04,700 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:57:04,744 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(951): ClusterId : 54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:57:04,744 DEBUG [RS:0;jenkins-hbase4:40293] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-17 16:57:04,746 DEBUG [RS:0;jenkins-hbase4:40293] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-17 16:57:04,746 DEBUG [RS:0;jenkins-hbase4:40293] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-17 16:57:04,749 DEBUG [RS:0;jenkins-hbase4:40293] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-17 16:57:04,751 DEBUG [RS:0;jenkins-hbase4:40293] zookeeper.ReadOnlyZKClient(139): Connect 0x70248c2f to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:57:04,755 DEBUG [RS:0;jenkins-hbase4:40293] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ee37c5b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:57:04,755 DEBUG [RS:0;jenkins-hbase4:40293] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7adce75f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:57:04,764 DEBUG [RS:0;jenkins-hbase4:40293] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:40293 2023-08-17 16:57:04,764 INFO [RS:0;jenkins-hbase4:40293] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-17 16:57:04,764 INFO [RS:0;jenkins-hbase4:40293] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-17 16:57:04,764 DEBUG [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1022): About to register with Master. 2023-08-17 16:57:04,764 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34623,1692291424497 with isa=jenkins-hbase4.apache.org/172.31.14.131:40293, startcode=1692291424529 2023-08-17 16:57:04,765 DEBUG [RS:0;jenkins-hbase4:40293] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-17 16:57:04,766 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60327, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-08-17 16:57:04,768 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34623] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:04,768 DEBUG [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4 2023-08-17 16:57:04,768 DEBUG [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41465 2023-08-17 16:57:04,768 DEBUG [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-17 16:57:04,772 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410029, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:04,772 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410026, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:04,772 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:57:04,772 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:57:04,772 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:04,772 DEBUG [RS:0;jenkins-hbase4:40293] zookeeper.ZKUtil(162): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:04,772 WARN [RS:0;jenkins-hbase4:40293] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-17 16:57:04,772 INFO [RS:0;jenkins-hbase4:40293] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:57:04,773 INFO [RS:0;jenkins-hbase4:40293] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:57:04,773 DEBUG [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:04,774 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40293,1692291424529] 2023-08-17 16:57:04,777 DEBUG [RS:0;jenkins-hbase4:40293] zookeeper.ZKUtil(162): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:04,778 DEBUG [RS:0;jenkins-hbase4:40293] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:57:04,778 INFO [RS:0;jenkins-hbase4:40293] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-17 16:57:04,779 INFO [RS:0;jenkins-hbase4:40293] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-17 16:57:04,779 INFO [RS:0;jenkins-hbase4:40293] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-17 16:57:04,779 INFO [RS:0;jenkins-hbase4:40293] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:04,780 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-17 16:57:04,781 INFO [RS:0;jenkins-hbase4:40293] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:04,782 DEBUG [RS:0;jenkins-hbase4:40293] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,782 DEBUG [RS:0;jenkins-hbase4:40293] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,782 DEBUG [RS:0;jenkins-hbase4:40293] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,782 DEBUG [RS:0;jenkins-hbase4:40293] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,782 DEBUG [RS:0;jenkins-hbase4:40293] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,782 DEBUG [RS:0;jenkins-hbase4:40293] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-17 16:57:04,782 DEBUG [RS:0;jenkins-hbase4:40293] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,782 DEBUG [RS:0;jenkins-hbase4:40293] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,782 DEBUG [RS:0;jenkins-hbase4:40293] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,782 DEBUG [RS:0;jenkins-hbase4:40293] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-17 16:57:04,783 INFO [RS:0;jenkins-hbase4:40293] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:04,783 INFO [RS:0;jenkins-hbase4:40293] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:04,783 INFO [RS:0;jenkins-hbase4:40293] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:04,793 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=101ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-17 16:57:04,794 INFO [RS:0;jenkins-hbase4:40293] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-17 16:57:04,794 INFO [RS:0;jenkins-hbase4:40293] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40293,1692291424529-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:04,805 INFO [RS:0;jenkins-hbase4:40293] regionserver.Replication(203): jenkins-hbase4.apache.org,40293,1692291424529 started 2023-08-17 16:57:04,805 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,40293,1692291424529, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40293, sessionid=0x1001b2c4f410033 2023-08-17 16:57:04,805 DEBUG [RS:0;jenkins-hbase4:40293] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-17 16:57:04,805 DEBUG [RS:0;jenkins-hbase4:40293] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:04,805 DEBUG [RS:0;jenkins-hbase4:40293] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40293,1692291424529' 2023-08-17 16:57:04,805 DEBUG [RS:0;jenkins-hbase4:40293] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-17 16:57:04,805 DEBUG [RS:0;jenkins-hbase4:40293] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-17 16:57:04,806 DEBUG [RS:0;jenkins-hbase4:40293] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-17 16:57:04,806 DEBUG [RS:0;jenkins-hbase4:40293] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-17 16:57:04,806 DEBUG [RS:0;jenkins-hbase4:40293] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:04,806 DEBUG [RS:0;jenkins-hbase4:40293] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40293,1692291424529' 2023-08-17 16:57:04,806 DEBUG [RS:0;jenkins-hbase4:40293] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-17 16:57:04,806 DEBUG [RS:0;jenkins-hbase4:40293] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-17 16:57:04,806 DEBUG [RS:0;jenkins-hbase4:40293] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-17 16:57:04,807 INFO [RS:0;jenkins-hbase4:40293] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-17 16:57:04,807 INFO [RS:0;jenkins-hbase4:40293] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-17 16:57:04,850 DEBUG [jenkins-hbase4:34623] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-17 16:57:04,851 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40293,1692291424529, state=OPENING 2023-08-17 16:57:04,853 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-17 16:57:04,853 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-17 16:57:04,853 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40293,1692291424529}] 2023-08-17 16:57:04,907 INFO [RS:0;jenkins-hbase4:40293] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:57:04,908 INFO [RS:0;jenkins-hbase4:40293] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40293%2C1692291424529.jenkins-hbase4.apache.org%2C40293%2C1692291424529.regiongroup-0, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,40293,1692291424529, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:57:04,925 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:57:04,925 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:57:04,925 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK] 2023-08-17 16:57:04,927 INFO [RS:0;jenkins-hbase4:40293] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,40293,1692291424529/jenkins-hbase4.apache.org%2C40293%2C1692291424529.jenkins-hbase4.apache.org%2C40293%2C1692291424529.regiongroup-0.1692291424909 2023-08-17 16:57:04,927 DEBUG [RS:0;jenkins-hbase4:40293] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK], DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK], DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK]] 2023-08-17 16:57:05,009 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:05,009 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:57:05,010 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48974, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-17 16:57:05,014 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-17 16:57:05,014 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:57:05,014 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:57:05,014 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:57:05,015 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40293%2C1692291424529.meta, suffix=.meta, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,40293,1692291424529, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:57:05,027 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:57:05,027 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:57:05,027 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK] 2023-08-17 16:57:05,029 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,40293,1692291424529/jenkins-hbase4.apache.org%2C40293%2C1692291424529.meta.1692291425016.meta 2023-08-17 16:57:05,029 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK], DatanodeInfoWithStorage[127.0.0.1:33417,DS-ec9ac215-be61-4c0f-b81f-2c4ad2bc1db7,DISK], DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK]] 2023-08-17 16:57:05,029 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:57:05,030 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-17 16:57:05,030 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-17 16:57:05,030 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-17 16:57:05,030 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-17 16:57:05,030 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:57:05,030 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-17 16:57:05,030 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-17 16:57:05,031 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-17 16:57:05,032 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info 2023-08-17 16:57:05,032 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info 2023-08-17 16:57:05,032 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-17 16:57:05,038 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info/4dee513191c14004ab80c6f18c55997a 2023-08-17 16:57:05,041 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info/d862d25bca50496b8832b846a15b761e 2023-08-17 16:57:05,041 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:05,042 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-17 16:57:05,042 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:57:05,043 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/rep_barrier 2023-08-17 16:57:05,043 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-17 16:57:05,044 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:05,044 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-17 16:57:05,045 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table 2023-08-17 16:57:05,045 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table 2023-08-17 16:57:05,045 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-17 16:57:05,049 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/table/0a63d2c9dbf3474c8afb858114c8a01a 2023-08-17 16:57:05,050 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:05,050 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740 2023-08-17 16:57:05,051 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740 2023-08-17 16:57:05,053 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-17 16:57:05,054 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-17 16:57:05,055 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72494124, jitterRate=0.08024662733078003}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-17 16:57:05,055 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-17 16:57:05,056 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1692291425009 2023-08-17 16:57:05,060 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-17 16:57:05,060 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-17 16:57:05,061 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40293,1692291424529, state=OPEN 2023-08-17 16:57:05,062 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-17 16:57:05,062 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-17 16:57:05,064 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-08-17 16:57:05,064 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40293,1692291424529 in 209 msec 2023-08-17 16:57:05,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-08-17 16:57:05,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 367 msec 2023-08-17 16:57:06,295 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1603ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-08-17 16:57:07,340 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-17 16:57:07,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3105ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3004ms 2023-08-17 16:57:09,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-08-17 16:57:09,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-17 16:57:09,203 DEBUG [hconnection-0x52cbd0be-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:57:09,204 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49820, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:57:09,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=37b9df605a0d48d3a0fcac9d8ab4c0c4, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34993,1692291390151, regionLocation=jenkins-hbase4.apache.org,34993,1692291390151, openSeqNum=10 2023-08-17 16:57:09,212 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=7b7c5dd31d075846c031f622b5c28bc1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34993,1692291390151, regionLocation=jenkins-hbase4.apache.org,34993,1692291390151, openSeqNum=10 2023-08-17 16:57:09,212 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ec395d682a88bce1d4eae00d5f8e72c7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34993,1692291390151, regionLocation=jenkins-hbase4.apache.org,34993,1692291390151, openSeqNum=12 2023-08-17 16:57:09,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-17 16:57:09,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692291489212 2023-08-17 16:57:09,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692291549212 2023-08-17 16:57:09,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 12 msec 2023-08-17 16:57:09,227 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34993,1692291390151 had 4 regions 2023-08-17 16:57:09,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34623,1692291424497-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:09,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34623,1692291424497-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:09,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34623,1692291424497-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:09,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:34623, period=300000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:09,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-17 16:57:09,228 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. is NOT online; state={37b9df605a0d48d3a0fcac9d8ab4c0c4 state=OPEN, ts=1692291429212, server=jenkins-hbase4.apache.org,34993,1692291390151}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-08-17 16:57:09,229 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34993,1692291390151, splitWal=true, meta=true, isMeta: false 2023-08-17 16:57:09,231 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151-splitting dir is empty, no logs to split. 2023-08-17 16:57:09,232 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,34993,1692291390151 WAL count=0, meta=false 2023-08-17 16:57:09,232 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,34993,1692291390151/hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4., unknown_server=jenkins-hbase4.apache.org,34993,1692291390151/t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1., unknown_server=jenkins-hbase4.apache.org,34993,1692291390151/t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:09,234 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,34993,1692291390151-splitting dir is empty, no logs to split. 2023-08-17 16:57:09,234 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,34993,1692291390151 WAL count=0, meta=false 2023-08-17 16:57:09,234 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34993,1692291390151 WAL splitting is done? wals=0, meta=false 2023-08-17 16:57:09,237 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN}] 2023-08-17 16:57:09,238 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN 2023-08-17 16:57:09,238 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN 2023-08-17 16:57:09,238 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN 2023-08-17 16:57:09,238 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:57:09,239 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:57:09,239 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-17 16:57:09,389 DEBUG [jenkins-hbase4:34623] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-17 16:57:09,393 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=37b9df605a0d48d3a0fcac9d8ab4c0c4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:09,393 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=ec395d682a88bce1d4eae00d5f8e72c7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:09,393 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=7b7c5dd31d075846c031f622b5c28bc1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:09,393 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291429391"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291429391"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291429391"}]},"ts":"1692291429391"} 2023-08-17 16:57:09,393 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291429393"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291429393"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291429393"}]},"ts":"1692291429393"} 2023-08-17 16:57:09,393 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291429393"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692291429393"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692291429393"}]},"ts":"1692291429393"} 2023-08-17 16:57:09,395 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 37b9df605a0d48d3a0fcac9d8ab4c0c4, server=jenkins-hbase4.apache.org,40293,1692291424529}] 2023-08-17 16:57:09,396 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure ec395d682a88bce1d4eae00d5f8e72c7, server=jenkins-hbase4.apache.org,40293,1692291424529}] 2023-08-17 16:57:09,399 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=30, state=RUNNABLE; OpenRegionProcedure 7b7c5dd31d075846c031f622b5c28bc1, server=jenkins-hbase4.apache.org,40293,1692291424529}] 2023-08-17 16:57:09,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:09,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-17 16:57:09,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40293%2C1692291424529.jenkins-hbase4.apache.org%2C40293%2C1692291424529.regiongroup-1, suffix=, logDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,40293,1692291424529, archiveDir=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs, maxLogs=10 2023-08-17 16:57:09,555 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:09,555 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 37b9df605a0d48d3a0fcac9d8ab4c0c4, NAME => 'hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:57:09,555 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:57:09,556 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:57:09,556 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:57:09,556 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:57:09,557 INFO [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:57:09,558 DEBUG [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/info 2023-08-17 16:57:09,558 DEBUG [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/info 2023-08-17 16:57:09,558 INFO [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 37b9df605a0d48d3a0fcac9d8ab4c0c4 columnFamilyName info 2023-08-17 16:57:09,568 DEBUG [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/info/8bfbb4041ee748bc8dba5ffabe3bdbca 2023-08-17 16:57:09,568 INFO [StoreOpener-37b9df605a0d48d3a0fcac9d8ab4c0c4-1] regionserver.HStore(310): Store=37b9df605a0d48d3a0fcac9d8ab4c0c4/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:09,569 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:57:09,569 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK] 2023-08-17 16:57:09,569 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK] 2023-08-17 16:57:09,569 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK] 2023-08-17 16:57:09,570 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:57:09,573 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:57:09,574 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 37b9df605a0d48d3a0fcac9d8ab4c0c4; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74117533, jitterRate=0.10443730652332306}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-17 16:57:09,574 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 37b9df605a0d48d3a0fcac9d8ab4c0c4: 2023-08-17 16:57:09,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/WALs/jenkins-hbase4.apache.org,40293,1692291424529/jenkins-hbase4.apache.org%2C40293%2C1692291424529.jenkins-hbase4.apache.org%2C40293%2C1692291424529.regiongroup-1.1692291429555 2023-08-17 16:57:09,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35599,DS-f4bf0c2b-01d3-46a5-893f-12d683e9310d,DISK], DatanodeInfoWithStorage[127.0.0.1:41833,DS-f4a589e1-b6f6-42d8-9012-b432d943742e,DISK], DatanodeInfoWithStorage[127.0.0.1:32949,DS-e8951513-3ad7-491e-a0b7-a7be4759f0f2,DISK]] 2023-08-17 16:57:09,578 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4., pid=33, masterSystemTime=1692291429550 2023-08-17 16:57:09,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7b7c5dd31d075846c031f622b5c28bc1, NAME => 't1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:57:09,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:57:09,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:57:09,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:57:09,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:57:09,581 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:09,581 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:09,581 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:57:09,581 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=37b9df605a0d48d3a0fcac9d8ab4c0c4, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:09,581 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692291429581"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291429581"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291429581"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291429581"}]},"ts":"1692291429581"} 2023-08-17 16:57:09,582 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1 2023-08-17 16:57:09,582 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1 2023-08-17 16:57:09,582 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b7c5dd31d075846c031f622b5c28bc1 columnFamilyName cf1 2023-08-17 16:57:09,584 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-08-17 16:57:09,584 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 37b9df605a0d48d3a0fcac9d8ab4c0c4, server=jenkins-hbase4.apache.org,40293,1692291424529 in 187 msec 2023-08-17 16:57:09,585 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=37b9df605a0d48d3a0fcac9d8ab4c0c4, ASSIGN in 349 msec 2023-08-17 16:57:09,593 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1/36b4336e2c014c019e4c7d9f163363d2 2023-08-17 16:57:09,600 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fb92a896eb74831961608bba2eef14b 2023-08-17 16:57:09,600 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1/6fb92a896eb74831961608bba2eef14b 2023-08-17 16:57:09,600 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(310): Store=7b7c5dd31d075846c031f622b5c28bc1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:09,600 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:57:09,601 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/norep 2023-08-17 16:57:09,601 DEBUG [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/norep 2023-08-17 16:57:09,602 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b7c5dd31d075846c031f622b5c28bc1 columnFamilyName norep 2023-08-17 16:57:09,602 INFO [StoreOpener-7b7c5dd31d075846c031f622b5c28bc1-1] regionserver.HStore(310): Store=7b7c5dd31d075846c031f622b5c28bc1/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:09,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:57:09,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:57:09,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-17 16:57:09,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:57:09,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7b7c5dd31d075846c031f622b5c28bc1; next sequenceid=17; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70795659, jitterRate=0.05493752658367157}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-17 16:57:09,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7b7c5dd31d075846c031f622b5c28bc1: 2023-08-17 16:57:09,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1., pid=35, masterSystemTime=1692291429550 2023-08-17 16:57:09,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:09,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:09,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:09,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ec395d682a88bce1d4eae00d5f8e72c7, NAME => 't2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.', STARTKEY => '', ENDKEY => ''} 2023-08-17 16:57:09,611 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=7b7c5dd31d075846c031f622b5c28bc1, regionState=OPEN, openSeqNum=17, regionLocation=jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:09,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:09,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-17 16:57:09,611 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291429611"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291429611"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291429611"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291429611"}]},"ts":"1692291429611"} 2023-08-17 16:57:09,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:09,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:09,613 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:09,614 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1 2023-08-17 16:57:09,614 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1 2023-08-17 16:57:09,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=30 2023-08-17 16:57:09,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=30, state=SUCCESS; OpenRegionProcedure 7b7c5dd31d075846c031f622b5c28bc1, server=jenkins-hbase4.apache.org,40293,1692291424529 in 216 msec 2023-08-17 16:57:09,614 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec395d682a88bce1d4eae00d5f8e72c7 columnFamilyName cf1 2023-08-17 16:57:09,615 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=7b7c5dd31d075846c031f622b5c28bc1, ASSIGN in 379 msec 2023-08-17 16:57:09,620 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0fcdfc43fa4d4bd8bcfb1fb4a19d2447 2023-08-17 16:57:09,621 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1/0fcdfc43fa4d4bd8bcfb1fb4a19d2447 2023-08-17 16:57:09,625 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(539): loaded hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1/5a8be9b14e044fb4be92342ffecc0175 2023-08-17 16:57:09,625 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(310): Store=ec395d682a88bce1d4eae00d5f8e72c7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:09,626 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:09,627 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/norep 2023-08-17 16:57:09,627 DEBUG [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/norep 2023-08-17 16:57:09,627 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec395d682a88bce1d4eae00d5f8e72c7 columnFamilyName norep 2023-08-17 16:57:09,628 INFO [StoreOpener-ec395d682a88bce1d4eae00d5f8e72c7-1] regionserver.HStore(310): Store=ec395d682a88bce1d4eae00d5f8e72c7/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-17 16:57:09,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:09,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:09,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-17 16:57:09,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:09,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ec395d682a88bce1d4eae00d5f8e72c7; next sequenceid=19; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69789915, jitterRate=0.03995077311992645}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-17 16:57:09,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ec395d682a88bce1d4eae00d5f8e72c7: 2023-08-17 16:57:09,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7., pid=34, masterSystemTime=1692291429550 2023-08-17 16:57:09,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:09,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:09,637 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=ec395d682a88bce1d4eae00d5f8e72c7, regionState=OPEN, openSeqNum=19, regionLocation=jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:09,637 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1692291429637"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692291429637"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692291429637"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692291429637"}]},"ts":"1692291429637"} 2023-08-17 16:57:09,640 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-08-17 16:57:09,640 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure ec395d682a88bce1d4eae00d5f8e72c7, server=jenkins-hbase4.apache.org,40293,1692291424529 in 242 msec 2023-08-17 16:57:09,641 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=27 2023-08-17 16:57:09,642 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34993,1692291390151 after splitting done 2023-08-17 16:57:09,642 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=ec395d682a88bce1d4eae00d5f8e72c7, ASSIGN in 405 msec 2023-08-17 16:57:09,642 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34993,1692291390151 from processing; numProcessing=0 2023-08-17 16:57:09,643 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34993,1692291390151, splitWal=true, meta=true in 4.9920 sec 2023-08-17 16:57:09,805 INFO [jenkins-hbase4:40293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:57:10,229 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-08-17 16:57:10,241 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-08-17 16:57:10,244 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-08-17 16:57:10,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.694sec 2023-08-17 16:57:10,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-17 16:57:10,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-17 16:57:10,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-17 16:57:10,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34623,1692291424497-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-17 16:57:10,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34623,1692291424497-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-17 16:57:10,248 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(139): Connect 0x0eef6066 to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:57:10,248 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-17 16:57:10,253 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@ca47bf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:57:10,254 DEBUG [hconnection-0x7b202aaf-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:57:10,255 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49824, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:57:10,257 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-08-17 16:57:10,258 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0eef6066 to 127.0.0.1:56119 2023-08-17 16:57:10,258 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:10,259 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:34623 after: jenkins-hbase4.apache.org:34623 2023-08-17 16:57:10,259 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(139): Connect 0x57b3bf2c to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:57:10,265 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d3c8227, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:57:10,767 DEBUG [hconnection-0x27142824-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:57:10,768 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49826, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-17 16:57:10,778 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-08-17 16:57:10,779 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-17 16:57:10,779 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-17 16:57:10,780 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-08-17 16:57:10,788 INFO [Listener at localhost/35639] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1692291430788 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:57:10,791 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication16922914307880x0, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:57:10,792 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1692291430788-0x1001b2c4f410038 connected Start Replication Server start 2023-08-17 16:57:10,795 INFO [Listener at localhost/35639] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-17 16:57:10,795 INFO [Listener at localhost/35639] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-17 16:57:10,807 INFO [Listener at localhost/35639] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-17 16:57:10,808 DEBUG [Listener at localhost/35639] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-17 16:57:10,818 INFO [Listener at localhost/35639] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-17 16:57:10,819 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,37377,1692291416917] other RSs: [] 2023-08-17 16:57:10,838 DEBUG [Listener at localhost/35639.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x7e72b6aa to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:57:10,842 DEBUG [Listener at localhost/35639.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1ebe5339, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:57:10,842 INFO [Listener at localhost/35639.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:57:10,844 DEBUG [Listener at localhost/35639.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:57:10,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1001b2c4f41003a connected 2023-08-17 16:57:10,846 INFO [Listener at localhost/35639.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=1067016f-f973-4161-8f76-366ce33cec43 to cluster=54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:57:14,804 INFO [jenkins-hbase4:40293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:57:14,813 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,37377,1692291416917/1's WALs to 1692291430795.syncuptool.replication.org,1234,1 2023-08-17 16:57:14,814 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1.1692291421836 with data PBUF\x08\x97\x06 2023-08-17 16:57:14,815 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0.1692291417331 with data PBUF\x08\xC6\x03 2023-08-17 16:57:14,818 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,37377,1692291416917/1's WALs to 1692291430795.syncuptool.replication.org,1234,1 2023-08-17 16:57:15,058 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-17 16:57:16,819 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,37377,1692291416917] 2023-08-17 16:57:16,820 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,37377,1692291416917] 2023-08-17 16:57:16,830 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,37377,1692291416917] 2023-08-17 16:57:16,830 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,37377,1692291416917, ReplicationSource: 1, currentBandwidth=0 2023-08-17 16:57:16,830 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:16,851 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917] zookeeper.ReadOnlyZKClient(139): Connect 0x1bc3859e to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:57:16,855 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@19fc394b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:57:16,855 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56119 2023-08-17 16:57:16,857 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-17 16:57:16,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1001b2c4f41003c connected 2023-08-17 16:57:16,859 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,37377,1692291416917 (queues=2) is replicating from cluster=1067016f-f973-4161-8f76-366ce33cec43 to cluster=54bb0ae5-8b51-4448-bfbd-06d7b08b9f06 2023-08-17 16:57:16,859 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0 2023-08-17 16:57:16,861 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,37377,1692291416917, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:57:16,861 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1 2023-08-17 16:57:16,861 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0 2023-08-17 16:57:16,863 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,37377,1692291416917, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-17 16:57:16,864 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1 2023-08-17 16:57:16,875 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1.1692291421836 2023-08-17 16:57:16,875 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceWALReader(162): Read 100 WAL entries eligible for replication 2023-08-17 16:57:16,875 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-17 16:57:16,876 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,37377,1692291416917 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4b04d98f902897a06d2d5c032dc77d72/112=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/113=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/114=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/115=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/116=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/117=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/118=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/119=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/120=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/121=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/122=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/123=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/124=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/125=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/126=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/127=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/128=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/129=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/130=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/131=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/132=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/133=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/134=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/135=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/136=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/137=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/138=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/139=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/140=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/141=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/142=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/143=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/144=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/145=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/146=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/147=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/148=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/149=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/150=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/151=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/152=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/153=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/154=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/155=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/156=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/157=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/158=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/159=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/160=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/161=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/162=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/163=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/164=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/165=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/166=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/167=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/168=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/169=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/170=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/171=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/172=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/173=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/174=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/175=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/176=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/177=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/178=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/179=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/180=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/181=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/182=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/183=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/184=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/185=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/186=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/187=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/188=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/189=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/190=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/191=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/192=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/193=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/194=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/195=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/196=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/197=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/198=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/199=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/200=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/201=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/202=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/203=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/204=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/205=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/206=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/207=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/208=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/209=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/210=[#edits: 1 = ],208}, {t1_syncup/4b04d98f902897a06d2d5c032dc77d72/211=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1.1692291421836, lastWalPosition=12631, nbRowKeys=100, nbHFiles=0, heapSize=20800, lastSeqIds={}, endOfFile=true,usedBufferSize=20800] 2023-08-17 16:57:16,878 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0.1692291417331 2023-08-17 16:57:16,878 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceWALReader(162): Read 200 WAL entries eligible for replication 2023-08-17 16:57:16,880 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-17 16:57:16,882 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-17 16:57:16,882 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,37377,1692291416917 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/a2668222e397f37ad3389946a7fe032f/212=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/213=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/214=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/215=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/216=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/217=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/218=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/219=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/220=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/221=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/222=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/223=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/224=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/225=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/226=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/227=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/228=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/229=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/230=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/231=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/232=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/233=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/234=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/235=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/236=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/237=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/238=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/239=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/240=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/241=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/242=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/243=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/244=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/245=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/246=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/247=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/248=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/249=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/250=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/251=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/252=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/253=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/254=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/255=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/256=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/257=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/258=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/259=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/260=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/261=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/262=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/263=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/264=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/265=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/266=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/267=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/268=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/269=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/270=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/271=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/272=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/273=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/274=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/275=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/276=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/277=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/278=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/279=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/280=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/281=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/282=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/283=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/284=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/285=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/286=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/287=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/288=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/289=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/290=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/291=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/292=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/293=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/294=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/295=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/296=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/297=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/298=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/299=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/300=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/301=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/302=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/303=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/304=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/305=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/306=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/307=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/308=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/309=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/310=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/311=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/312=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/313=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/314=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/315=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/316=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/317=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/318=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/319=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/320=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/321=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/322=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/323=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/324=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/325=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/326=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/327=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/328=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/329=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/330=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/331=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/332=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/333=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/334=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/335=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/336=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/337=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/338=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/339=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/340=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/341=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/342=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/343=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/344=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/345=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/346=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/347=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/348=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/349=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/350=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/351=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/352=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/353=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/354=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/355=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/356=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/357=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/358=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/359=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/360=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/361=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/362=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/363=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/364=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/365=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/366=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/367=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/368=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/369=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/370=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/371=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/372=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/373=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/374=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/375=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/376=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/377=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/378=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/379=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/380=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/381=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/382=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/383=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/384=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/385=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/386=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/387=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/388=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/389=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/390=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/391=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/392=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/393=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/394=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/395=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/396=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/397=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/398=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/399=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/400=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/401=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/402=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/403=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/404=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/405=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/406=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/407=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/408=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/409=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/410=[#edits: 1 = ],208}, {t2_syncup/a2668222e397f37ad3389946a7fe032f/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38511/user/jenkins/test-data/788a8271-4cc9-5920-47b3-eb9fe6878b85/oldWALs/jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0.1692291417331, lastWalPosition=22804, nbRowKeys=200, nbHFiles=0, heapSize=41600, lastSeqIds={}, endOfFile=true,usedBufferSize=41600] 2023-08-17 16:57:16,883 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35618, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-17 16:57:16,886 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40293] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-17 16:57:16,887 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40293] zookeeper.ReadOnlyZKClient(139): Connect 0x6289bf6a to 127.0.0.1:56119 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-17 16:57:16,889 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=40293] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-17 16:57:16,891 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40293] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@6c4e5d4b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-17 16:57:16,894 DEBUG [hconnection-0x6871d41a-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-17 16:57:16,895 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35622, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-08-17 16:57:16,908 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40293] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-17 16:57:16,909 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,37377,1692291416917] 2023-08-17 16:57:16,910 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1.1692291421836] 2023-08-17 16:57:16,913 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=40293] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-17 16:57:16,914 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,37377,1692291416917] 2023-08-17 16:57:16,914 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,37377,1692291416917 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:57:16,914 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:16,916 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0.1692291417331] 2023-08-17 16:57:16,918 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,37377,1692291416917 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-17 16:57:16,918 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:17,014 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:17,017 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication1692291430788-0x1001b2c4f410038, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1692291430795.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:17,017 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-1,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,37377,1692291416917 with the following stats: Total replicated edits: 300, current progress: 2023-08-17 16:57:17,117 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,37377,1692291416917 2023-08-17 16:57:17,117 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] zookeeper.ZKUtil(243): syncupReplication1692291430788-0x1001b2c4f410038, quorum=127.0.0.1:56119, baseZNode=/1 Unable to list children of znode /1/replication/rs/1692291430795.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,37377,1692291416917 because node does not exist (not an error) 2023-08-17 16:57:17,118 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917.replicationSource.shipperjenkins-hbase4.apache.org%2C37377%2C1692291416917.jenkins-hbase4.apache.org%2C37377%2C1692291416917.regiongroup-0,1-jenkins-hbase4.apache.org,37377,1692291416917] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,37377,1692291416917 with the following stats: Total replicated edits: 300, current progress: 2023-08-17 16:57:19,804 INFO [jenkins-hbase4:40293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-17 16:57:19,805 INFO [jenkins-hbase4:40293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-08-17 16:57:20,819 INFO [Listener at localhost/35639] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-17 16:57:20,923 DEBUG [Listener at localhost/35639.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f41003a, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:20,923 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e72b6aa to 127.0.0.1:56119 2023-08-17 16:57:20,924 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:21,130 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication1692291430788-0x1001b2c4f410038, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:21,130 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): syncupReplication1692291430788-0x1001b2c4f410038, quorum=127.0.0.1:56119, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:21,139 INFO [Listener at localhost/35639] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-08-17 16:57:21,139 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-17 16:57:21,139 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57b3bf2c to 127.0.0.1:56119 2023-08-17 16:57:21,139 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:21,139 DEBUG [Listener at localhost/35639] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-17 16:57:21,139 DEBUG [Listener at localhost/35639] util.JVMClusterUtil(257): Found active master hash=1836316956, stopped=false 2023-08-17 16:57:21,139 INFO [Listener at localhost/35639] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,34623,1692291424497 2023-08-17 16:57:21,141 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-17 16:57:21,142 INFO [Listener at localhost/35639] procedure2.ProcedureExecutor(629): Stopping 2023-08-17 16:57:21,141 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-17 16:57:21,142 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:57:21,143 DEBUG [Listener at localhost/35639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6aa846b2 to 127.0.0.1:56119 2023-08-17 16:57:21,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:57:21,144 DEBUG [Listener at localhost/35639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:21,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-17 16:57:21,144 INFO [Listener at localhost/35639] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40293,1692291424529' ***** 2023-08-17 16:57:21,145 INFO [Listener at localhost/35639] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-17 16:57:21,145 INFO [RS:0;jenkins-hbase4:40293] regionserver.HeapMemoryManager(220): Stopping 2023-08-17 16:57:21,145 INFO [RS:0;jenkins-hbase4:40293] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-17 16:57:21,145 INFO [RS:0;jenkins-hbase4:40293] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-17 16:57:21,145 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(3305): Received CLOSE for 7b7c5dd31d075846c031f622b5c28bc1 2023-08-17 16:57:21,145 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-17 16:57:21,146 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(3305): Received CLOSE for ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:21,146 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(3305): Received CLOSE for 37b9df605a0d48d3a0fcac9d8ab4c0c4 2023-08-17 16:57:21,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7b7c5dd31d075846c031f622b5c28bc1, disabling compactions & flushes 2023-08-17 16:57:21,147 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:21,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:21,147 DEBUG [RS:0;jenkins-hbase4:40293] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x70248c2f to 127.0.0.1:56119 2023-08-17 16:57:21,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:21,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. after waiting 0 ms 2023-08-17 16:57:21,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:21,147 DEBUG [RS:0;jenkins-hbase4:40293] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:21,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 7b7c5dd31d075846c031f622b5c28bc1 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-08-17 16:57:21,147 INFO [RS:0;jenkins-hbase4:40293] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-17 16:57:21,148 INFO [RS:0;jenkins-hbase4:40293] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-17 16:57:21,148 INFO [RS:0;jenkins-hbase4:40293] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-17 16:57:21,148 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-17 16:57:21,148 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-17 16:57:21,148 DEBUG [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1478): Online Regions={7b7c5dd31d075846c031f622b5c28bc1=t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1., ec395d682a88bce1d4eae00d5f8e72c7=t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7., 1588230740=hbase:meta,,1.1588230740, 37b9df605a0d48d3a0fcac9d8ab4c0c4=hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4.} 2023-08-17 16:57:21,149 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-17 16:57:21,149 DEBUG [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1504): Waiting on 1588230740, 37b9df605a0d48d3a0fcac9d8ab4c0c4, 7b7c5dd31d075846c031f622b5c28bc1, ec395d682a88bce1d4eae00d5f8e72c7 2023-08-17 16:57:21,149 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-17 16:57:21,150 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-17 16:57:21,150 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-17 16:57:21,150 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-17 16:57:21,150 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-08-17 16:57:21,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=20 (bloomFilter=true), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/.tmp/cf1/be1e77aece9b47c1aaced7a6739489bf 2023-08-17 16:57:21,184 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/.tmp/info/a6a75265d26a4c2f80cc3187f497fe6c 2023-08-17 16:57:21,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/.tmp/cf1/be1e77aece9b47c1aaced7a6739489bf as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1/be1e77aece9b47c1aaced7a6739489bf 2023-08-17 16:57:21,193 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/.tmp/info/a6a75265d26a4c2f80cc3187f497fe6c as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info/a6a75265d26a4c2f80cc3187f497fe6c 2023-08-17 16:57:21,196 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:57:21,200 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/info/a6a75265d26a4c2f80cc3187f497fe6c, entries=24, sequenceid=43, filesize=7.4 K 2023-08-17 16:57:21,200 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/cf1/be1e77aece9b47c1aaced7a6739489bf, entries=100, sequenceid=20, filesize=8.3 K 2023-08-17 16:57:21,203 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 53ms, sequenceid=43, compaction requested=true 2023-08-17 16:57:21,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 7b7c5dd31d075846c031f622b5c28bc1 in 54ms, sequenceid=20, compaction requested=true 2023-08-17 16:57:21,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t1_syncup/7b7c5dd31d075846c031f622b5c28bc1/recovered.edits/23.seqid, newMaxSeqId=23, maxSeqId=16 2023-08-17 16:57:21,217 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-08-17 16:57:21,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7b7c5dd31d075846c031f622b5c28bc1: 2023-08-17 16:57:21,218 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-17 16:57:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1692291384362.7b7c5dd31d075846c031f622b5c28bc1. 2023-08-17 16:57:21,219 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-17 16:57:21,219 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-17 16:57:21,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ec395d682a88bce1d4eae00d5f8e72c7, disabling compactions & flushes 2023-08-17 16:57:21,220 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:21,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:21,220 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-17 16:57:21,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. after waiting 0 ms 2023-08-17 16:57:21,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:21,220 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ec395d682a88bce1d4eae00d5f8e72c7 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-08-17 16:57:21,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/.tmp/cf1/e0d976dcb658477eb332dab06ca80e44 2023-08-17 16:57:21,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/.tmp/cf1/e0d976dcb658477eb332dab06ca80e44 as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1/e0d976dcb658477eb332dab06ca80e44 2023-08-17 16:57:21,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/cf1/e0d976dcb658477eb332dab06ca80e44, entries=200, sequenceid=22, filesize=12.2 K 2023-08-17 16:57:21,243 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for ec395d682a88bce1d4eae00d5f8e72c7 in 23ms, sequenceid=22, compaction requested=true 2023-08-17 16:57:21,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/default/t2_syncup/ec395d682a88bce1d4eae00d5f8e72c7/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=18 2023-08-17 16:57:21,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:21,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ec395d682a88bce1d4eae00d5f8e72c7: 2023-08-17 16:57:21,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1692291384976.ec395d682a88bce1d4eae00d5f8e72c7. 2023-08-17 16:57:21,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 37b9df605a0d48d3a0fcac9d8ab4c0c4, disabling compactions & flushes 2023-08-17 16:57:21,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:21,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:21,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. after waiting 0 ms 2023-08-17 16:57:21,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:21,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/data/hbase/namespace/37b9df605a0d48d3a0fcac9d8ab4c0c4/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-08-17 16:57:21,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:21,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 37b9df605a0d48d3a0fcac9d8ab4c0c4: 2023-08-17 16:57:21,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692291382507.37b9df605a0d48d3a0fcac9d8ab4c0c4. 2023-08-17 16:57:21,349 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40293,1692291424529; all regions closed. 2023-08-17 16:57:21,355 DEBUG [RS:0;jenkins-hbase4:40293] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:57:21,355 INFO [RS:0;jenkins-hbase4:40293] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40293%2C1692291424529.meta:.meta(num 1692291425016) 2023-08-17 16:57:21,360 DEBUG [RS:0;jenkins-hbase4:40293] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:57:21,360 INFO [RS:0;jenkins-hbase4:40293] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40293%2C1692291424529.jenkins-hbase4.apache.org%2C40293%2C1692291424529.regiongroup-1:(num 1692291429555) 2023-08-17 16:57:21,365 DEBUG [RS:0;jenkins-hbase4:40293] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/oldWALs 2023-08-17 16:57:21,365 INFO [RS:0;jenkins-hbase4:40293] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40293%2C1692291424529.jenkins-hbase4.apache.org%2C40293%2C1692291424529.regiongroup-0:(num 1692291424909) 2023-08-17 16:57:21,365 DEBUG [RS:0;jenkins-hbase4:40293] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:21,365 INFO [RS:0;jenkins-hbase4:40293] regionserver.LeaseManager(133): Closed leases 2023-08-17 16:57:21,365 INFO [RS:0;jenkins-hbase4:40293] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-17 16:57:21,365 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:57:21,365 DEBUG [RS:0;jenkins-hbase4:40293] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6289bf6a to 127.0.0.1:56119 2023-08-17 16:57:21,365 DEBUG [RS:0;jenkins-hbase4:40293] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:21,366 INFO [RS:0;jenkins-hbase4:40293] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40293 2023-08-17 16:57:21,371 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:21,371 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410026, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:21,371 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,40293,1692291424529 2023-08-17 16:57:21,371 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:57:21,371 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f410029, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:21,371 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:21,371 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:57:21,371 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1001b2c4f41003c, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-17 16:57:21,372 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-17 16:57:21,373 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40293,1692291424529] 2023-08-17 16:57:21,373 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40293,1692291424529; numProcessing=1 2023-08-17 16:57:21,374 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,40293,1692291424529 already deleted, retry=false 2023-08-17 16:57:21,374 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40293,1692291424529 expired; onlineServers=0 2023-08-17 16:57:21,374 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34623,1692291424497' ***** 2023-08-17 16:57:21,374 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-17 16:57:21,375 DEBUG [M:0;jenkins-hbase4:34623] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@694c606a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-17 16:57:21,375 INFO [M:0;jenkins-hbase4:34623] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34623,1692291424497 2023-08-17 16:57:21,375 INFO [M:0;jenkins-hbase4:34623] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34623,1692291424497; all regions closed. 2023-08-17 16:57:21,375 DEBUG [M:0;jenkins-hbase4:34623] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-17 16:57:21,375 DEBUG [M:0;jenkins-hbase4:34623] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-17 16:57:21,375 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-17 16:57:21,375 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291424688] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692291424688,5,FailOnTimeoutGroup] 2023-08-17 16:57:21,375 DEBUG [M:0;jenkins-hbase4:34623] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-17 16:57:21,375 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291424692] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692291424692,5,FailOnTimeoutGroup] 2023-08-17 16:57:21,377 INFO [M:0;jenkins-hbase4:34623] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-17 16:57:21,377 INFO [M:0;jenkins-hbase4:34623] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-17 16:57:21,377 INFO [M:0;jenkins-hbase4:34623] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-17 16:57:21,377 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-08-17 16:57:21,377 DEBUG [M:0;jenkins-hbase4:34623] master.HMaster(1512): Stopping service threads 2023-08-17 16:57:21,377 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-17 16:57:21,377 INFO [M:0;jenkins-hbase4:34623] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-17 16:57:21,378 ERROR [M:0;jenkins-hbase4:34623] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-17 16:57:21,378 INFO [M:0;jenkins-hbase4:34623] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-17 16:57:21,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-17 16:57:21,378 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-17 16:57:21,379 DEBUG [M:0;jenkins-hbase4:34623] zookeeper.ZKUtil(398): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-08-17 16:57:21,379 WARN [M:0;jenkins-hbase4:34623] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-17 16:57:21,379 INFO [M:0;jenkins-hbase4:34623] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-17 16:57:21,379 INFO [M:0;jenkins-hbase4:34623] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-17 16:57:21,380 DEBUG [M:0;jenkins-hbase4:34623] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-17 16:57:21,380 INFO [M:0;jenkins-hbase4:34623] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:21,380 DEBUG [M:0;jenkins-hbase4:34623] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:21,380 DEBUG [M:0;jenkins-hbase4:34623] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-17 16:57:21,380 DEBUG [M:0;jenkins-hbase4:34623] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:21,380 INFO [M:0;jenkins-hbase4:34623] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-08-17 16:57:21,393 INFO [M:0;jenkins-hbase4:34623] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=309 (bloomFilter=true), to=hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d6f0dcdfe7734f02a63460518bdcafe6 2023-08-17 16:57:21,397 DEBUG [M:0;jenkins-hbase4:34623] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d6f0dcdfe7734f02a63460518bdcafe6 as hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d6f0dcdfe7734f02a63460518bdcafe6 2023-08-17 16:57:21,401 INFO [M:0;jenkins-hbase4:34623] regionserver.HStore(1080): Added hdfs://localhost:41465/user/jenkins/test-data/233113df-6901-e0f9-687c-5326da7a16a4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d6f0dcdfe7734f02a63460518bdcafe6, entries=9, sequenceid=309, filesize=5.5 K 2023-08-17 16:57:21,402 INFO [M:0;jenkins-hbase4:34623] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=309, compaction requested=true 2023-08-17 16:57:21,403 INFO [M:0;jenkins-hbase4:34623] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-17 16:57:21,403 DEBUG [M:0;jenkins-hbase4:34623] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-17 16:57:21,406 INFO [M:0;jenkins-hbase4:34623] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-17 16:57:21,406 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-17 16:57:21,407 INFO [M:0;jenkins-hbase4:34623] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34623 2023-08-17 16:57:21,410 DEBUG [M:0;jenkins-hbase4:34623] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,34623,1692291424497 already deleted, retry=false 2023-08-17 16:57:21,542 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:21,542 INFO [M:0;jenkins-hbase4:34623] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34623,1692291424497; zookeeper connection closed. 2023-08-17 16:57:21,542 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): master:34623-0x1001b2c4f410032, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:21,642 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:21,642 INFO [RS:0;jenkins-hbase4:40293] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40293,1692291424529; zookeeper connection closed. 2023-08-17 16:57:21,642 DEBUG [Listener at localhost/35639-EventThread] zookeeper.ZKWatcher(600): regionserver:40293-0x1001b2c4f410033, quorum=127.0.0.1:56119, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-17 16:57:21,642 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@67f8b4b4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@67f8b4b4 2023-08-17 16:57:21,642 INFO [Listener at localhost/35639] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-17 16:57:21,643 WARN [Listener at localhost/35639] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-17 16:57:21,647 INFO [Listener at localhost/35639] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-17 16:57:21,753 WARN [BP-162676127-172.31.14.131-1692291380682 heartbeating to localhost/127.0.0.1:41465] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-17 16:57:21,753 WARN [BP-162676127-172.31.14.131-1692291380682 heartbeating to localhost/127.0.0.1:41465] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-162676127-172.31.14.131-1692291380682 (Datanode Uuid 04fbd196-147c-4b53-9a23-f2beaf7b81e0) service to localhost/127.0.0.1:41465 2023-08-17 16:57:21,754 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/cluster_8b57984d-197b-73e4-852f-61f49906a62d/dfs/data/data7/current/BP-162676127-172.31.14.131-1692291380682] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:21,755 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/cluster_8b57984d-197b-73e4-852f-61f49906a62d/dfs/data/data8/current/BP-162676127-172.31.14.131-1692291380682] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:21,758 WARN [Listener at localhost/35639] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-17 16:57:21,761 INFO [Listener at localhost/35639] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-17 16:57:21,867 WARN [BP-162676127-172.31.14.131-1692291380682 heartbeating to localhost/127.0.0.1:41465] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-17 16:57:21,868 WARN [BP-162676127-172.31.14.131-1692291380682 heartbeating to localhost/127.0.0.1:41465] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-162676127-172.31.14.131-1692291380682 (Datanode Uuid 240f4026-adbd-43ba-9d5b-6ae01f75beb8) service to localhost/127.0.0.1:41465 2023-08-17 16:57:21,868 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/cluster_8b57984d-197b-73e4-852f-61f49906a62d/dfs/data/data5/current/BP-162676127-172.31.14.131-1692291380682] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:21,869 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/cluster_8b57984d-197b-73e4-852f-61f49906a62d/dfs/data/data6/current/BP-162676127-172.31.14.131-1692291380682] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:21,870 WARN [Listener at localhost/35639] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-17 16:57:21,873 INFO [Listener at localhost/35639] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-17 16:57:21,978 WARN [BP-162676127-172.31.14.131-1692291380682 heartbeating to localhost/127.0.0.1:41465] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-17 16:57:21,978 WARN [BP-162676127-172.31.14.131-1692291380682 heartbeating to localhost/127.0.0.1:41465] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-162676127-172.31.14.131-1692291380682 (Datanode Uuid fa6903f1-cccf-4423-8dd2-e495823c58d9) service to localhost/127.0.0.1:41465 2023-08-17 16:57:21,978 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/cluster_8b57984d-197b-73e4-852f-61f49906a62d/dfs/data/data3/current/BP-162676127-172.31.14.131-1692291380682] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:21,979 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/cluster_8b57984d-197b-73e4-852f-61f49906a62d/dfs/data/data4/current/BP-162676127-172.31.14.131-1692291380682] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:21,980 WARN [Listener at localhost/35639] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-17 16:57:21,983 INFO [Listener at localhost/35639] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-17 16:57:22,088 WARN [BP-162676127-172.31.14.131-1692291380682 heartbeating to localhost/127.0.0.1:41465] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-17 16:57:22,088 WARN [BP-162676127-172.31.14.131-1692291380682 heartbeating to localhost/127.0.0.1:41465] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-162676127-172.31.14.131-1692291380682 (Datanode Uuid f04c7764-baa1-403e-aa86-0841390ce624) service to localhost/127.0.0.1:41465 2023-08-17 16:57:22,088 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/cluster_8b57984d-197b-73e4-852f-61f49906a62d/dfs/data/data1/current/BP-162676127-172.31.14.131-1692291380682] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:22,089 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c0eb7210-f2fc-28e6-9520-2c551e04e7b8/cluster_8b57984d-197b-73e4-852f-61f49906a62d/dfs/data/data2/current/BP-162676127-172.31.14.131-1692291380682] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:22,115 INFO [Listener at localhost/35639] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-17 16:57:22,293 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-17 16:57:22,293 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-17 16:57:22,293 WARN [Listener at localhost/35639] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-17 16:57:22,295 INFO [Listener at localhost/35639] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-17 16:57:22,396 WARN [BP-1083969836-172.31.14.131-1692291374046 heartbeating to localhost/127.0.0.1:38511] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-17 16:57:22,396 WARN [BP-1083969836-172.31.14.131-1692291374046 heartbeating to localhost/127.0.0.1:38511] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1083969836-172.31.14.131-1692291374046 (Datanode Uuid 173bd245-09c0-4c76-92f9-36a923215fca) service to localhost/127.0.0.1:38511 2023-08-17 16:57:22,397 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/cluster_72c0133a-0ba1-29b3-3528-a3510554313d/dfs/data/data3/current/BP-1083969836-172.31.14.131-1692291374046] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:22,397 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/cluster_72c0133a-0ba1-29b3-3528-a3510554313d/dfs/data/data4/current/BP-1083969836-172.31.14.131-1692291374046] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:22,397 WARN [Listener at localhost/35639] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-17 16:57:22,399 INFO [Listener at localhost/35639] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-17 16:57:22,500 WARN [BP-1083969836-172.31.14.131-1692291374046 heartbeating to localhost/127.0.0.1:38511] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-17 16:57:22,500 WARN [BP-1083969836-172.31.14.131-1692291374046 heartbeating to localhost/127.0.0.1:38511] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1083969836-172.31.14.131-1692291374046 (Datanode Uuid e47db782-26c3-4869-b653-86aa503a6c38) service to localhost/127.0.0.1:38511 2023-08-17 16:57:22,501 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/cluster_72c0133a-0ba1-29b3-3528-a3510554313d/dfs/data/data1/current/BP-1083969836-172.31.14.131-1692291374046] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:22,501 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b91d0dcc-aea1-f4c4-4a30-dc843af885e1/cluster_72c0133a-0ba1-29b3-3528-a3510554313d/dfs/data/data2/current/BP-1083969836-172.31.14.131-1692291374046] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-17 16:57:22,509 INFO [Listener at localhost/35639] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-17 16:57:22,516 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-17 16:57:22,624 INFO [Listener at localhost/35639] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-17 16:57:22,648 INFO [Listener at localhost/35639] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-17 16:57:22,669 INFO [Listener at localhost/35639] hbase.ResourceChecker(175): after: replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL#testSyncUpTool Thread=189 (was 10) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x6774d1fc-SendThread(127.0.0.1:56119) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1232191603) connection to localhost/127.0.0.1:38511 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-16 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1232191603) connection to localhost/127.0.0.1:38511 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: IPC Client (1232191603) connection to localhost/127.0.0.1:41465 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:38511 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:38511 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x1bc3859e-SendThread(127.0.0.1:56119) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x62c3242c-SendThread(127.0.0.1:56119) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/35639 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053-SendThread(127.0.0.1:56119) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x460e10e6-SendThread(127.0.0.1:56119) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x7974c63d-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:38511 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:38511 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1232191603) connection to localhost/127.0.0.1:41465 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x460e10e6-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@61dfd54f java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:41465 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:41465 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x1bc3859e sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/72152296.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x62c3242c-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x6774d1fc-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:41465 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,37377,1692291416917-SendThread(127.0.0.1:56119) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:41465 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x6774d1fc sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/72152296.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1232191603) connection to localhost/127.0.0.1:41465 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45379,1692291378053-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x62c3242c sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/72152296.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x7974c63d sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/72152296.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:41465 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x460e10e6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/72152296.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x1bc3859e-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:41465 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:41465 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56119@0x7974c63d-SendThread(127.0.0.1:56119) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42469,1692291378095-SendThread(127.0.0.1:56119) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) - Thread LEAK? -, OpenFileDescriptor=670 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=78 (was 175), ProcessCount=171 (was 170) - ProcessCount LEAK? -, AvailableMemoryMB=4002 (was 5184)